The following values were not passed to `accelerate launch` and had defaults used instead:
		More than one GPU was found, enabling multi-GPU training.
		If this was unintended please pass in `--num_processes=1`.
	`--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
2026-04-30:10:53:43 INFO     [_cli.run:358] [rank=4] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO     [_cli.run:358] [rank=6] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO     [_cli.run:358] [rank=1] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO     [_cli.run:358] [rank=3] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO     [_cli.run:358] [rank=5] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO     [_cli.run:358] [rank=7] non-main process detected, suppressing wandb logging.
2026-04-30:10:53:43 INFO     [_cli.run:358] [rank=2] non-main process detected, suppressing wandb logging.
wandb: [wandb.login()] Loaded credentials for https://api.wandb.ai from WANDB_API_KEY.
wandb: Currently logged in as: kkhya (maskmoe) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.26.1
wandb: Run data is saved locally in /nfs/ywang29/lm-factory/wandb/run-20260430_105344-jm4o6xvg
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run fft_coding
wandb: ⭐️ View project at https://wandb.ai/maskmoe/MFT-LM-eval
wandb: 🚀 View run at https://wandb.ai/maskmoe/MFT-LM-eval/runs/jm4o6xvg
2026-04-30:10:54:46 INFO     [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO     [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO     [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO     [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO     [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO     [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO     [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:46 INFO     [_cli.run:388] Selected Tasks: ['humaneval', 'humaneval_plus', 'mbpp', 'mbpp_plus']
2026-04-30:10:54:47 INFO     [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO     [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO     [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO     [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO     [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO     [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO     [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO     [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO     [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO     [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO     [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO     [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO     [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO     [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:54:47 INFO     [evaluator:213] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
2026-04-30:10:54:47 INFO     [evaluator:238] Initializing hf model, with arguments: {'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'trust_remote_code': True, 'max_length': 4096}
2026-04-30:10:55:00 INFO     [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO     [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO     [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO     [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO     [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO     [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO     [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:00 INFO     [models.huggingface:273] Using `accelerate launch` or `parallelize=True`, device 'cuda:0' will be overridden when placing model.
2026-04-30:10:55:01 INFO     [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:2'}
2026-04-30:10:55:02 INFO     [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:3'}
2026-04-30:10:55:02 INFO     [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:7'}
2026-04-30:10:55:02 INFO     [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:4'}
2026-04-30:10:55:02 INFO     [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:1'}
2026-04-30:10:55:02 INFO     [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:6'}
2026-04-30:10:55:02 INFO     [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:5'}
2026-04-30:10:55:02 INFO     [models.huggingface:524] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}

Loading weights:   0%|          | 0/443 [00:00<?, ?it/s]
Loading weights:   0%|          | 0/443 [00:00<?, ?it/s]
Loading weights:   0%|          | 0/443 [00:00<?, ?it/s]
Loading weights:   0%|          | 0/443 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/443 [00:00<00:00, 8289.14it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 1/443 [00:00<00:00, 5282.50it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 1/443 [00:00<00:00, 8473.34it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 1/443 [00:00<00:00, 6159.04it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 1/443 [00:00<00:00, 5391.14it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 1/443 [00:00<00:00, 4275.54it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 0/443 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/443 [00:00<00:00, 13443.28it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 1/443 [00:00<00:00, 8886.24it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 1/443 [00:00<00:00, 5562.74it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 0/443 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/443 [00:00<00:00, 3134.76it/s, Materializing param=lm_head.weight] 
Loading weights:   0%|          | 1/443 [00:00<00:00, 9383.23it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 1/443 [00:00<00:00, 5874.38it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 0/443 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/443 [00:00<00:00, 8035.07it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 1/443 [00:00<00:00, 5184.55it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 0/443 [00:00<?, ?it/s]
Loading weights:   0%|          | 1/443 [00:00<00:00, 8065.97it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 1/443 [00:00<00:00, 4718.00it/s, Materializing param=lm_head.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=lm_head.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=lm_head.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=lm_head.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=lm_head.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=lm_head.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=lm_head.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=lm_head.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:14,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:15,  9.56s/it, Materializing param=lm_head.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:15,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   0%|          | 2/443 [00:19<1:10:15,  9.56s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|          | 3/443 [00:19<1:10:05,  9.56s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 3/443 [00:19<1:10:05,  9.56s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 4/443 [00:19<1:09:56,  9.56s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 4/443 [00:19<1:09:56,  9.56s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|          | 5/443 [00:20<24:15,  3.32s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 5/443 [00:20<24:15,  3.32s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|          | 5/443 [00:20<24:15,  3.32s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:21<19:55,  2.74s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:21<19:55,  2.74s/it, Materializing param=model.layers.0.mlp.up_proj.weight]  
Loading weights:   1%|▏         | 6/443 [00:21<19:55,  2.74s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:22<16:22,  2.25s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:22<16:22,  2.25s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 7/443 [00:22<16:22,  2.25s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 8/443 [00:22<16:20,  2.25s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]        
Loading weights:   2%|▏         | 8/443 [00:22<16:20,  2.25s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights:   2%|▏         | 9/443 [00:22<16:18,  2.25s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 9/443 [00:22<16:18,  2.25s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:22<16:15,  2.25s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:22<16:15,  2.25s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:22<06:59,  1.03it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:22<06:59,  1.03it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   2%|▏         | 11/443 [00:22<06:59,  1.03it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   3%|▎         | 12/443 [00:22<06:58,  1.03it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 12/443 [00:22<06:58,  1.03it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:22<05:07,  1.40it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:22<05:07,  1.40it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:22<05:07,  1.40it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   1%|          | 3/443 [00:22<52:15,  7.13s/it, Materializing param=model.embed_tokens.weight]  
Loading weights:   1%|          | 3/443 [00:22<52:15,  7.13s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 3/443 [00:22<52:15,  7.13s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 4/443 [00:22<52:08,  7.13s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 4/443 [00:22<52:08,  7.13s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   3%|▎         | 14/443 [00:22<05:06,  1.40it/s, Materializing param=model.layers.1.input_layernorm.weight] 
Loading weights:   3%|▎         | 14/443 [00:22<05:06,  1.40it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   3%|▎         | 15/443 [00:22<05:06,  1.40it/s, Materializing param=model.layers.1.mlp.down_proj.weight]  
Loading weights:   3%|▎         | 15/443 [00:22<05:06,  1.40it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   1%|          | 3/443 [00:23<52:53,  7.21s/it, Materializing param=model.embed_tokens.weight]  
Loading weights:   1%|          | 3/443 [00:23<52:53,  7.21s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 3/443 [00:23<52:53,  7.21s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 4/443 [00:23<52:46,  7.21s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 4/443 [00:23<52:46,  7.21s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|          | 5/443 [00:23<24:41,  3.38s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|          | 5/443 [00:23<24:41,  3.38s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|          | 5/443 [00:23<24:41,  3.38s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|          | 5/443 [00:23<24:55,  3.41s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|          | 5/443 [00:23<24:55,  3.41s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|          | 5/443 [00:23<24:55,  3.41s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:23<18:21,  2.52s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:23<18:21,  2.52s/it, Materializing param=model.layers.0.mlp.up_proj.weight]  
Loading weights:   1%|▏         | 6/443 [00:23<18:21,  2.52s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:23<18:36,  2.55s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:23<18:36,  2.55s/it, Materializing param=model.layers.0.mlp.up_proj.weight]  
Loading weights:   1%|▏         | 6/443 [00:23<18:36,  2.55s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:24<13:51,  1.91s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:24<13:51,  1.91s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 7/443 [00:24<13:51,  1.91s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 8/443 [00:24<13:49,  1.91s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]        
Loading weights:   2%|▏         | 8/443 [00:24<13:49,  1.91s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights:   2%|▏         | 9/443 [00:24<13:47,  1.91s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 9/443 [00:24<13:47,  1.91s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:24<04:15,  1.67it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:24<04:15,  1.67it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:24<04:15,  1.67it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:24<13:45,  1.91s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:24<13:45,  1.91s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:24<14:01,  1.93s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:24<14:01,  1.93s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 7/443 [00:24<14:01,  1.93s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 8/443 [00:24<13:59,  1.93s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]        
Loading weights:   2%|▏         | 8/443 [00:24<13:59,  1.93s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights:   2%|▏         | 9/443 [00:24<13:57,  1.93s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 9/443 [00:24<13:57,  1.93s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:24<13:55,  1.93s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:24<13:55,  1.93s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:24<05:26,  1.32it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:24<05:26,  1.32it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   2%|▏         | 11/443 [00:24<05:26,  1.32it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   3%|▎         | 12/443 [00:24<05:25,  1.32it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 12/443 [00:24<05:25,  1.32it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:24<05:36,  1.28it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:24<05:36,  1.28it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   2%|▏         | 11/443 [00:24<05:36,  1.28it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   3%|▎         | 12/443 [00:24<05:36,  1.28it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 12/443 [00:24<05:36,  1.28it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:24<05:35,  1.28it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:24<05:35,  1.28it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 14/443 [00:24<05:34,  1.28it/s, Materializing param=model.layers.1.input_layernorm.weight] 
Loading weights:   3%|▎         | 14/443 [00:24<05:34,  1.28it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   3%|▎         | 15/443 [00:24<05:33,  1.28it/s, Materializing param=model.layers.1.mlp.down_proj.weight]  
Loading weights:   3%|▎         | 15/443 [00:24<05:33,  1.28it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:24<03:59,  1.80it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:24<03:59,  1.80it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:24<03:59,  1.80it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 14/443 [00:24<03:58,  1.80it/s, Materializing param=model.layers.1.input_layernorm.weight] 
Loading weights:   3%|▎         | 14/443 [00:24<03:58,  1.80it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   4%|▍         | 17/443 [00:24<04:27,  1.59it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:24<04:27,  1.59it/s, Materializing param=model.layers.1.mlp.up_proj.weight]  
Loading weights:   4%|▍         | 17/443 [00:24<04:27,  1.59it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   3%|▎         | 15/443 [00:25<03:21,  2.13it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   3%|▎         | 15/443 [00:25<03:21,  2.13it/s, Materializing param=model.layers.1.mlp.down_proj.weight]  
Loading weights:   3%|▎         | 15/443 [00:25<03:21,  2.13it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:25<03:20,  2.13it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:25<03:20,  2.13it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   1%|          | 3/443 [00:25<59:26,  8.11s/it, Materializing param=model.embed_tokens.weight]  
Loading weights:   1%|          | 3/443 [00:25<59:26,  8.11s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 3/443 [00:25<59:26,  8.11s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 4/443 [00:25<59:18,  8.11s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 4/443 [00:25<59:18,  8.11s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:25<03:14,  2.19it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:25<03:14,  2.19it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:25<03:14,  2.19it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:25<04:29,  1.58it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:25<04:29,  1.58it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 18/443 [00:25<04:29,  1.58it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 19/443 [00:25<04:28,  1.58it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]        
Loading weights:   4%|▍         | 19/443 [00:25<04:28,  1.58it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights:   5%|▍         | 20/443 [00:25<04:28,  1.58it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 20/443 [00:25<04:28,  1.58it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:25<04:27,  1.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:25<04:27,  1.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   1%|          | 3/443 [00:25<1:00:42,  8.28s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|          | 3/443 [00:25<1:00:42,  8.28s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 3/443 [00:25<1:00:42,  8.28s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 4/443 [00:25<1:00:34,  8.28s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 4/443 [00:25<1:00:34,  8.28s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:25<03:11,  2.23it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:25<03:11,  2.23it/s, Materializing param=model.layers.1.mlp.up_proj.weight]  
Loading weights:   4%|▍         | 17/443 [00:25<03:11,  2.23it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   1%|          | 3/443 [00:25<1:00:48,  8.29s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|          | 3/443 [00:25<1:00:48,  8.29s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|          | 3/443 [00:25<1:00:48,  8.29s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 3/443 [00:25<1:00:48,  8.29s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 3/443 [00:25<1:00:48,  8.29s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 3/443 [00:25<1:00:48,  8.29s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 4/443 [00:25<1:00:40,  8.29s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 4/443 [00:25<1:00:40,  8.29s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|          | 4/443 [00:25<1:00:40,  8.29s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 4/443 [00:25<1:00:40,  8.29s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:25<03:07,  2.28it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:25<03:07,  2.28it/s, Materializing param=model.layers.1.mlp.up_proj.weight]  
Loading weights:   4%|▍         | 17/443 [00:25<03:07,  2.28it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:25<02:28,  2.83it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:25<02:28,  2.83it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▍         | 22/443 [00:25<02:28,  2.83it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▌         | 23/443 [00:25<02:28,  2.83it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 23/443 [00:25<02:28,  2.83it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   1%|          | 5/443 [00:25<28:35,  3.92s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|          | 5/443 [00:25<28:35,  3.92s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|          | 5/443 [00:25<28:35,  3.92s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:26<02:06,  3.30it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:26<02:06,  3.30it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:26<02:06,  3.30it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   6%|▌         | 25/443 [00:26<02:06,  3.30it/s, Materializing param=model.layers.2.input_layernorm.weight] 
Loading weights:   6%|▌         | 25/443 [00:26<02:06,  3.30it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   1%|          | 5/443 [00:26<29:02,  3.98s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 5/443 [00:26<29:02,  3.98s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|          | 5/443 [00:26<29:02,  3.98s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:26<03:29,  2.03it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:26<03:29,  2.03it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 18/443 [00:26<03:29,  2.03it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 19/443 [00:26<03:28,  2.03it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]        
Loading weights:   4%|▍         | 19/443 [00:26<03:28,  2.03it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights:   5%|▍         | 20/443 [00:26<03:28,  2.03it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 20/443 [00:26<03:28,  2.03it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   1%|          | 5/443 [00:26<29:04,  3.98s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 5/443 [00:26<29:04,  3.98s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|          | 5/443 [00:26<29:04,  3.98s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:26<03:27,  2.03it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:26<03:27,  2.03it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   1%|          | 5/443 [00:26<29:06,  3.99s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 5/443 [00:26<29:06,  3.99s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|          | 5/443 [00:26<29:06,  3.99s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|          | 3/443 [00:26<1:03:18,  8.63s/it, Materializing param=model.embed_tokens.weight]
Loading weights:   1%|          | 3/443 [00:26<1:03:18,  8.63s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 3/443 [00:26<1:03:18,  8.63s/it, Materializing param=model.layers.0.input_layernorm.weight]
Loading weights:   1%|          | 4/443 [00:26<1:03:09,  8.63s/it, Materializing param=model.layers.0.mlp.down_proj.weight]  
Loading weights:   1%|          | 4/443 [00:26<1:03:09,  8.63s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
Loading weights:   1%|          | 5/443 [00:26<1:03:01,  8.63s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|          | 5/443 [00:26<1:03:01,  8.63s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:26<03:25,  2.07it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:26<03:25,  2.07it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 18/443 [00:26<03:25,  2.07it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 19/443 [00:26<03:25,  2.07it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]        
Loading weights:   4%|▍         | 19/443 [00:26<03:25,  2.07it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights:   5%|▍         | 20/443 [00:26<03:24,  2.07it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 20/443 [00:26<03:24,  2.07it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:26<01:56,  3.61it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:26<01:56,  3.61it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▍         | 22/443 [00:26<01:56,  3.61it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▌         | 23/443 [00:26<01:56,  3.61it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 23/443 [00:26<01:56,  3.61it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:26<03:24,  2.07it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:26<03:24,  2.07it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:26<01:36,  4.36it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:26<01:36,  4.36it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:26<01:36,  4.36it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:26<21:58,  3.02s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:26<21:58,  3.02s/it, Materializing param=model.layers.0.mlp.up_proj.weight]  
Loading weights:   1%|▏         | 6/443 [00:26<21:58,  3.02s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:26<01:53,  3.72it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:26<01:53,  3.72it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▍         | 22/443 [00:26<01:53,  3.72it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▌         | 23/443 [00:26<01:52,  3.72it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 23/443 [00:26<01:52,  3.72it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   6%|▌         | 25/443 [00:26<01:35,  4.36it/s, Materializing param=model.layers.2.input_layernorm.weight] 
Loading weights:   6%|▌         | 25/443 [00:26<01:35,  4.36it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   1%|▏         | 6/443 [00:26<21:37,  2.97s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:26<21:37,  2.97s/it, Materializing param=model.layers.0.mlp.up_proj.weight]  
Loading weights:   1%|▏         | 6/443 [00:26<21:37,  2.97s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:26<01:29,  4.67it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:26<01:29,  4.67it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:26<01:29,  4.67it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   6%|▌         | 25/443 [00:26<01:29,  4.67it/s, Materializing param=model.layers.2.input_layernorm.weight] 
Loading weights:   6%|▌         | 25/443 [00:26<01:29,  4.67it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   6%|▌         | 26/443 [00:26<01:29,  4.67it/s, Materializing param=model.layers.2.mlp.down_proj.weight]  
Loading weights:   6%|▌         | 26/443 [00:26<01:29,  4.67it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:27<16:33,  2.28s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:27<16:33,  2.28s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 7/443 [00:27<16:33,  2.28s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 8/443 [00:27<16:30,  2.28s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]        
Loading weights:   2%|▏         | 8/443 [00:27<16:30,  2.28s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights:   2%|▏         | 9/443 [00:27<16:28,  2.28s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 9/443 [00:27<16:28,  2.28s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:27<22:26,  3.08s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:27<22:26,  3.08s/it, Materializing param=model.layers.0.mlp.up_proj.weight]  
Loading weights:   1%|▏         | 6/443 [00:27<22:26,  3.08s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:27<22:26,  3.08s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:27<22:26,  3.08s/it, Materializing param=model.layers.0.mlp.up_proj.weight]  
Loading weights:   1%|▏         | 6/443 [00:27<22:26,  3.08s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:27<16:26,  2.28s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:27<16:26,  2.28s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   1%|▏         | 6/443 [00:27<23:43,  3.26s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]  
Loading weights:   1%|▏         | 6/443 [00:27<23:43,  3.26s/it, Materializing param=model.layers.0.mlp.up_proj.weight]  
Loading weights:   1%|▏         | 6/443 [00:27<23:43,  3.26s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:27<06:27,  1.12it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:27<06:27,  1.12it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   2%|▏         | 11/443 [00:27<06:27,  1.12it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   3%|▎         | 12/443 [00:27<06:26,  1.12it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 12/443 [00:27<06:26,  1.12it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:27<01:22,  5.02it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:27<01:22,  5.02it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:27<01:22,  5.02it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 26/443 [00:27<02:41,  2.58it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   6%|▌         | 26/443 [00:27<02:41,  2.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight]  
Loading weights:   6%|▌         | 26/443 [00:27<02:41,  2.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:27<02:41,  2.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:27<02:41,  2.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:27<04:38,  1.54it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:27<04:38,  1.54it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:27<04:38,  1.54it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   6%|▌         | 26/443 [00:27<01:56,  3.57it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   6%|▌         | 26/443 [00:27<01:56,  3.57it/s, Materializing param=model.layers.2.mlp.down_proj.weight]  
Loading weights:   6%|▌         | 26/443 [00:27<01:56,  3.57it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:27<01:56,  3.57it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:27<01:56,  3.57it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   3%|▎         | 14/443 [00:27<04:37,  1.54it/s, Materializing param=model.layers.1.input_layernorm.weight] 
Loading weights:   3%|▎         | 14/443 [00:27<04:37,  1.54it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   3%|▎         | 15/443 [00:27<04:37,  1.54it/s, Materializing param=model.layers.1.mlp.down_proj.weight]  
Loading weights:   3%|▎         | 15/443 [00:27<04:37,  1.54it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:27<17:07,  2.36s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:27<17:07,  2.36s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 7/443 [00:27<17:07,  2.36s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 8/443 [00:27<17:05,  2.36s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]        
Loading weights:   2%|▏         | 8/443 [00:27<17:05,  2.36s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights:   2%|▏         | 9/443 [00:27<17:03,  2.36s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 9/443 [00:27<17:03,  2.36s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:27<17:00,  2.36s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:27<17:00,  2.36s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:27<06:40,  1.08it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:27<06:40,  1.08it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   2%|▏         | 11/443 [00:27<06:40,  1.08it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   3%|▎         | 12/443 [00:27<06:39,  1.08it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 12/443 [00:27<06:39,  1.08it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:27<19:06,  2.63s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:27<19:06,  2.63s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 7/443 [00:27<19:06,  2.63s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 8/443 [00:27<19:04,  2.63s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]        
Loading weights:   2%|▏         | 8/443 [00:27<19:04,  2.63s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights:   2%|▏         | 9/443 [00:27<19:01,  2.63s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 9/443 [00:27<19:01,  2.63s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:27<18:59,  2.63s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:27<18:59,  2.63s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:27<17:25,  2.40s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:27<17:25,  2.40s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 7/443 [00:27<17:25,  2.40s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 8/443 [00:27<17:23,  2.40s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]        
Loading weights:   2%|▏         | 8/443 [00:27<17:23,  2.40s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights:   2%|▏         | 9/443 [00:27<17:20,  2.40s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 9/443 [00:27<17:20,  2.40s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:27<17:25,  2.40s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
Loading weights:   2%|▏         | 7/443 [00:27<17:25,  2.40s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 7/443 [00:27<17:25,  2.40s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
Loading weights:   2%|▏         | 8/443 [00:27<17:23,  2.40s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]        
Loading weights:   2%|▏         | 8/443 [00:27<17:23,  2.40s/it, Materializing param=model.layers.0.self_attn.k_norm.weight]
Loading weights:   2%|▏         | 9/443 [00:27<17:21,  2.40s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 9/443 [00:27<17:21,  2.40s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:27<17:18,  2.40s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:27<17:18,  2.40s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:27<17:18,  2.40s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 10/443 [00:27<17:18,  2.40s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:27<04:44,  1.51it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:27<04:44,  1.51it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:27<04:44,  1.51it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 14/443 [00:27<04:43,  1.51it/s, Materializing param=model.layers.1.input_layernorm.weight] 
Loading weights:   3%|▎         | 14/443 [00:27<04:43,  1.51it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   3%|▎         | 15/443 [00:27<04:43,  1.51it/s, Materializing param=model.layers.1.mlp.down_proj.weight]  
Loading weights:   3%|▎         | 15/443 [00:27<04:43,  1.51it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:27<08:15,  1.15s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:27<08:15,  1.15s/it, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   2%|▏         | 11/443 [00:27<08:15,  1.15s/it, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   3%|▎         | 12/443 [00:27<08:14,  1.15s/it, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 12/443 [00:27<08:14,  1.15s/it, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:27<06:43,  1.07it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:27<06:43,  1.07it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   2%|▏         | 11/443 [00:27<06:43,  1.07it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   3%|▎         | 12/443 [00:27<06:42,  1.07it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 12/443 [00:27<06:42,  1.07it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:16,  2.17it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:16,  2.17it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:16,  2.17it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:28<05:57,  1.20it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:28<05:57,  1.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:28<05:57,  1.20it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 14/443 [00:28<05:56,  1.20it/s, Materializing param=model.layers.1.input_layernorm.weight] 
Loading weights:   3%|▎         | 14/443 [00:28<05:56,  1.20it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   3%|▎         | 15/443 [00:28<05:55,  1.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight]  
Loading weights:   3%|▎         | 15/443 [00:28<05:55,  1.20it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:28<06:51,  1.05it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
Loading weights:   2%|▏         | 11/443 [00:28<06:51,  1.05it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   2%|▏         | 11/443 [00:28<06:51,  1.05it/s, Materializing param=model.layers.0.self_attn.q_norm.weight]
Loading weights:   3%|▎         | 12/443 [00:28<06:50,  1.05it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 12/443 [00:28<06:50,  1.05it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:28<06:49,  1.05it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:28<06:49,  1.05it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:28<04:46,  1.50it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:28<04:46,  1.50it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 13/443 [00:28<04:46,  1.50it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
Loading weights:   3%|▎         | 14/443 [00:28<06:48,  1.05it/s, Materializing param=model.layers.1.input_layernorm.weight] 
Loading weights:   3%|▎         | 14/443 [00:28<06:48,  1.05it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   3%|▎         | 15/443 [00:28<06:47,  1.05it/s, Materializing param=model.layers.1.mlp.down_proj.weight]  
Loading weights:   3%|▎         | 15/443 [00:28<06:47,  1.05it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   3%|▎         | 14/443 [00:28<04:45,  1.50it/s, Materializing param=model.layers.1.input_layernorm.weight] 
Loading weights:   3%|▎         | 14/443 [00:28<04:45,  1.50it/s, Materializing param=model.layers.1.input_layernorm.weight]
Loading weights:   3%|▎         | 15/443 [00:28<04:44,  1.50it/s, Materializing param=model.layers.1.mlp.down_proj.weight]  
Loading weights:   3%|▎         | 15/443 [00:28<04:44,  1.50it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:16,  2.17it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:16,  2.17it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:16,  2.17it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:28<03:11,  2.22it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:28<03:11,  2.22it/s, Materializing param=model.layers.1.mlp.up_proj.weight]  
Loading weights:   4%|▍         | 17/443 [00:28<03:11,  2.22it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<04:01,  1.77it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<04:01,  1.77it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<04:01,  1.77it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:33,  2.00it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:33,  2.00it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:33,  2.00it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:17,  2.16it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:17,  2.16it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▎         | 16/443 [00:28<03:17,  2.16it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:28<03:09,  2.25it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:28<03:09,  2.25it/s, Materializing param=model.layers.1.mlp.up_proj.weight]  
Loading weights:   4%|▍         | 17/443 [00:28<03:09,  2.25it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:28<03:08,  2.25it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:28<03:08,  2.25it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 18/443 [00:28<03:08,  2.25it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 19/443 [00:28<03:08,  2.25it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]        
Loading weights:   4%|▍         | 19/443 [00:28<03:08,  2.25it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights:   5%|▍         | 20/443 [00:28<03:07,  2.25it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 20/443 [00:28<03:07,  2.25it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   6%|▋         | 28/443 [00:28<02:39,  2.60it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▋         | 28/443 [00:28<02:41,  2.57it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▋         | 28/443 [00:28<03:19,  2.08it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▋         | 28/443 [00:28<02:39,  2.60it/s, Materializing param=model.layers.2.mlp.up_proj.weight]  
Loading weights:   6%|▋         | 28/443 [00:28<02:41,  2.57it/s, Materializing param=model.layers.2.mlp.up_proj.weight]  
Loading weights:   6%|▋         | 28/443 [00:28<02:39,  2.60it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   6%|▋         | 28/443 [00:28<03:19,  2.08it/s, Materializing param=model.layers.2.mlp.up_proj.weight]  
Loading weights:   6%|▋         | 28/443 [00:28<02:41,  2.57it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   6%|▋         | 28/443 [00:28<03:19,  2.08it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:28<03:07,  2.25it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:28<03:07,  2.25it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:28<04:01,  1.77it/s, Materializing param=model.layers.1.mlp.up_proj.weight]  
Loading weights:   4%|▍         | 17/443 [00:28<04:01,  1.77it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:28<03:26,  2.07it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:28<03:26,  2.07it/s, Materializing param=model.layers.1.mlp.up_proj.weight]  
Loading weights:   4%|▍         | 17/443 [00:28<03:26,  2.07it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▍         | 17/443 [00:28<03:17,  2.16it/s, Materializing param=model.layers.1.mlp.up_proj.weight]  
Loading weights:   4%|▍         | 17/443 [00:28<03:17,  2.16it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:28<01:36,  4.34it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:28<01:36,  4.34it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▍         | 22/443 [00:28<01:36,  4.34it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▌         | 23/443 [00:28<01:36,  4.34it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 23/443 [00:28<01:36,  4.34it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:16,  5.49it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:16,  5.49it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:16,  5.49it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:02,  2.33it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:02,  2.33it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:02,  2.33it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 19/443 [00:29<03:01,  2.33it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]        
Loading weights:   4%|▍         | 19/443 [00:29<03:01,  2.33it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights:   5%|▍         | 20/443 [00:29<03:01,  2.33it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 20/443 [00:29<03:01,  2.33it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   6%|▌         | 25/443 [00:29<01:16,  5.49it/s, Materializing param=model.layers.2.input_layernorm.weight] 
Loading weights:   6%|▌         | 25/443 [00:29<01:16,  5.49it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   6%|▌         | 26/443 [00:29<01:15,  5.49it/s, Materializing param=model.layers.2.mlp.down_proj.weight]  
Loading weights:   6%|▌         | 26/443 [00:29<01:15,  5.49it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:29<03:00,  2.33it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:29<03:00,  2.33it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:29<01:35,  4.40it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:29<01:35,  4.40it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▍         | 22/443 [00:29<01:35,  4.40it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▌         | 23/443 [00:29<01:35,  4.40it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 23/443 [00:29<01:35,  4.40it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:39,  1.94it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:39,  1.94it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:39,  1.94it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 19/443 [00:29<03:38,  1.94it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]        
Loading weights:   4%|▍         | 19/443 [00:29<03:38,  1.94it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights:   5%|▍         | 20/443 [00:29<03:38,  1.94it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 20/443 [00:29<03:38,  1.94it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:29<03:37,  1.94it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:29<03:37,  1.94it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:18,  2.14it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:18,  2.14it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:18,  2.14it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 19/443 [00:29<03:18,  2.14it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]        
Loading weights:   4%|▍         | 19/443 [00:29<03:18,  2.14it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights:   5%|▍         | 20/443 [00:29<03:17,  2.14it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 20/443 [00:29<03:17,  2.14it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:08,  2.26it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:08,  2.26it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 18/443 [00:29<03:08,  2.26it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
Loading weights:   4%|▍         | 19/443 [00:29<03:07,  2.26it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]        
Loading weights:   4%|▍         | 19/443 [00:29<03:07,  2.26it/s, Materializing param=model.layers.1.self_attn.k_norm.weight]
Loading weights:   5%|▍         | 20/443 [00:29<03:07,  2.26it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 20/443 [00:29<03:07,  2.26it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:17,  5.40it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:17,  5.40it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:17,  5.40it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:29<03:06,  2.26it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:29<03:06,  2.26it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:29<02:09,  3.25it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:29<02:09,  3.25it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▍         | 22/443 [00:29<02:09,  3.25it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▌         | 23/443 [00:29<02:09,  3.25it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 23/443 [00:29<02:09,  3.25it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   6%|▌         | 25/443 [00:29<01:17,  5.40it/s, Materializing param=model.layers.2.input_layernorm.weight] 
Loading weights:   6%|▌         | 25/443 [00:29<01:17,  5.40it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   6%|▌         | 26/443 [00:29<01:17,  5.40it/s, Materializing param=model.layers.2.mlp.down_proj.weight]  
Loading weights:   6%|▌         | 26/443 [00:29<01:17,  5.40it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:29<02:05,  3.35it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:29<02:05,  3.35it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 21/443 [00:29<02:05,  3.35it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:29<02:05,  3.35it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▍         | 22/443 [00:29<02:05,  3.35it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▌         | 23/443 [00:29<02:05,  3.35it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 23/443 [00:29<02:05,  3.35it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:29<01:51,  3.78it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
Loading weights:   5%|▍         | 22/443 [00:29<01:51,  3.78it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▍         | 22/443 [00:29<01:51,  3.78it/s, Materializing param=model.layers.1.self_attn.q_norm.weight]
Loading weights:   5%|▌         | 23/443 [00:29<01:51,  3.78it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 23/443 [00:29<01:51,  3.78it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:44,  4.01it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:44,  4.01it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:44,  4.01it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:29<01:11,  5.85it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:29<01:11,  5.85it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:29<01:11,  5.85it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 25/443 [00:29<01:44,  4.01it/s, Materializing param=model.layers.2.input_layernorm.weight] 
Loading weights:   6%|▌         | 25/443 [00:29<01:44,  4.01it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   6%|▌         | 26/443 [00:29<01:44,  4.01it/s, Materializing param=model.layers.2.mlp.down_proj.weight]  
Loading weights:   6%|▌         | 26/443 [00:29<01:44,  4.01it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:25,  4.92it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:25,  4.92it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:25,  4.92it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   6%|▌         | 25/443 [00:29<01:24,  4.92it/s, Materializing param=model.layers.2.input_layernorm.weight] 
Loading weights:   6%|▌         | 25/443 [00:29<01:24,  4.92it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   6%|▌         | 26/443 [00:29<01:24,  4.92it/s, Materializing param=model.layers.2.mlp.down_proj.weight]  
Loading weights:   6%|▌         | 26/443 [00:29<01:24,  4.92it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:29,  4.67it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:29,  4.67it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   5%|▌         | 24/443 [00:29<01:29,  4.67it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
Loading weights:   6%|▌         | 25/443 [00:29<01:29,  4.67it/s, Materializing param=model.layers.2.input_layernorm.weight] 
Loading weights:   6%|▌         | 25/443 [00:29<01:29,  4.67it/s, Materializing param=model.layers.2.input_layernorm.weight]
Loading weights:   6%|▌         | 26/443 [00:29<01:29,  4.67it/s, Materializing param=model.layers.2.mlp.down_proj.weight]  
Loading weights:   6%|▌         | 26/443 [00:29<01:29,  4.67it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:29<01:09,  5.96it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:29<01:09,  5.96it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:29<01:09,  5.96it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▋         | 28/443 [00:29<01:10,  5.85it/s, Materializing param=model.layers.2.mlp.up_proj.weight]  
Loading weights:   6%|▋         | 28/443 [00:29<01:10,  5.85it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:29<01:28,  4.69it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:29<01:28,  4.69it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:29<01:28,  4.69it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:30<01:18,  5.33it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:30<01:18,  5.33it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:30<01:18,  5.33it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:30<01:21,  5.11it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:30<01:21,  5.11it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▌         | 27/443 [00:30<01:21,  5.11it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
Loading weights:   6%|▋         | 28/443 [00:30<01:09,  5.96it/s, Materializing param=model.layers.2.mlp.up_proj.weight]  
Loading weights:   6%|▋         | 28/443 [00:30<01:09,  5.96it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   6%|▋         | 28/443 [00:30<01:28,  4.69it/s, Materializing param=model.layers.2.mlp.up_proj.weight]  
Loading weights:   6%|▋         | 28/443 [00:30<01:28,  4.69it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   6%|▋         | 28/443 [00:30<01:17,  5.33it/s, Materializing param=model.layers.2.mlp.up_proj.weight]  
Loading weights:   6%|▋         | 28/443 [00:30<01:17,  5.33it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   6%|▋         | 28/443 [00:30<01:21,  5.11it/s, Materializing param=model.layers.2.mlp.up_proj.weight]  
Loading weights:   6%|▋         | 28/443 [00:30<01:21,  5.11it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   7%|▋         | 29/443 [00:30<04:25,  1.56it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   7%|▋         | 29/443 [00:30<04:25,  1.56it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:32,  4.47it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:55,  3.57it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   7%|▋         | 29/443 [00:30<04:25,  1.56it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:32,  4.48it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:40,  4.11it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:45,  3.91it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:55,  3.57it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<04:53,  1.41it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   7%|▋         | 29/443 [00:30<04:18,  1.60it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
Loading weights:   7%|▋         | 30/443 [00:30<04:24,  1.56it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]        
Loading weights:   7%|▋         | 29/443 [00:30<01:40,  4.11it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:55,  3.57it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:45,  3.91it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<04:53,  1.41it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 30/443 [00:30<04:24,  1.56it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:40,  4.11it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:45,  3.91it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:32,  4.47it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<04:53,  1.41it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 30/443 [00:30<01:55,  3.57it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]        
Loading weights:   7%|▋         | 29/443 [00:30<01:32,  4.47it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<04:18,  1.60it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 30/443 [00:30<01:40,  4.11it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]        
Loading weights:   7%|▋         | 30/443 [00:30<01:45,  3.91it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]        
Loading weights:   7%|▋         | 30/443 [00:30<01:55,  3.57it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights:   7%|▋         | 30/443 [00:30<01:45,  3.91it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights:   7%|▋         | 30/443 [00:30<01:40,  4.11it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights:   7%|▋         | 29/443 [00:30<04:18,  1.60it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 31/443 [00:30<04:24,  1.56it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 31/443 [00:30<04:24,  1.56it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 31/443 [00:30<01:55,  3.57it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:32,  4.48it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 30/443 [00:30<01:32,  4.47it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]        
Loading weights:   7%|▋         | 31/443 [00:30<01:45,  3.91it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 31/443 [00:30<01:55,  3.57it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 31/443 [00:30<01:45,  3.91it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 31/443 [00:30<01:40,  4.11it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 30/443 [00:30<04:53,  1.41it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]        
Loading weights:   7%|▋         | 30/443 [00:30<01:32,  4.47it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights:   7%|▋         | 31/443 [00:30<01:40,  4.11it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 30/443 [00:30<04:53,  1.41it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights:   7%|▋         | 30/443 [00:30<04:17,  1.60it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]        
Loading weights:   7%|▋         | 30/443 [00:30<04:17,  1.60it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights:   7%|▋         | 31/443 [00:30<04:52,  1.41it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 31/443 [00:30<01:32,  4.47it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 31/443 [00:30<04:52,  1.41it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 31/443 [00:30<01:32,  4.47it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 31/443 [00:30<04:16,  1.60it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 31/443 [00:30<04:16,  1.60it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 29/443 [00:30<01:32,  4.48it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
Loading weights:   7%|▋         | 30/443 [00:30<01:32,  4.48it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]        
Loading weights:   7%|▋         | 30/443 [00:30<01:32,  4.48it/s, Materializing param=model.layers.2.self_attn.k_norm.weight]
Loading weights:   7%|▋         | 31/443 [00:30<01:31,  4.48it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 31/443 [00:30<01:31,  4.48it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:15,  5.47it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:06,  6.21it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:15,  5.47it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:06,  6.21it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:15,  5.47it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:06,  6.21it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<02:56,  2.32it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<02:38,  2.59it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:20,  5.12it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<02:38,  2.59it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<02:38,  2.59it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:20,  5.12it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:20,  5.12it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<02:56,  2.32it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:06,  6.20it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<02:56,  2.32it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:10,  5.81it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<02:37,  2.60it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:10,  5.81it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:10,  5.81it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<02:37,  2.60it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<02:37,  2.60it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:06,  6.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 32/443 [00:30<01:06,  6.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 33/443 [00:31<01:05,  6.21it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<01:05,  6.21it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<03:06,  2.20it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
Loading weights:   7%|▋         | 33/443 [00:31<01:14,  5.47it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<01:20,  5.12it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<02:38,  2.59it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<01:10,  5.81it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<01:14,  5.47it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<01:06,  6.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<03:06,  2.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<02:37,  2.60it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<01:20,  5.12it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<03:06,  2.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<01:10,  5.81it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<02:37,  2.60it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   7%|▋         | 33/443 [00:31<02:38,  2.59it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:25,  4.80it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:29,  4.57it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:25,  4.80it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<03:05,  2.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<02:25,  2.81it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:25,  4.80it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:29,  4.57it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:22,  4.97it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   8%|▊         | 34/443 [00:31<03:05,  2.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:29,  4.57it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<02:25,  2.81it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<02:25,  2.81it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:22,  4.97it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:22,  4.97it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   7%|▋         | 33/443 [00:31<01:06,  6.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:18,  5.20it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:18,  5.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:18,  5.20it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<02:26,  2.79it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   8%|▊         | 34/443 [00:31<02:26,  2.79it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<02:26,  2.79it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<02:26,  2.79it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<02:26,  2.79it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<01:50,  3.69it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<01:50,  3.69it/s, Materializing param=model.layers.3.input_layernorm.weight] 
Loading weights:   8%|▊         | 36/443 [00:31<01:50,  3.69it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:   8%|▊         | 37/443 [00:31<01:50,  3.69it/s, Materializing param=model.layers.3.mlp.down_proj.weight]  
Loading weights:   8%|▊         | 37/443 [00:31<01:50,  3.69it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<01:22,  4.97it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<01:22,  4.97it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:28,  4.64it/s, Materializing param=model.layers.2.self_attn.q_norm.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:28,  4.64it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 34/443 [00:31<01:28,  4.64it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<01:27,  4.64it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<01:27,  4.64it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<02:14,  3.04it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<02:18,  2.95it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<02:14,  3.04it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<02:14,  3.04it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<02:18,  2.95it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<02:18,  2.95it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<01:29,  4.57it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<01:29,  4.57it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<01:25,  4.80it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<01:25,  4.80it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<01:18,  5.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 35/443 [00:31<01:18,  5.20it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<01:12,  5.65it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<01:12,  5.65it/s, Materializing param=model.layers.3.input_layernorm.weight] 
Loading weights:   8%|▊         | 36/443 [00:31<01:12,  5.65it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:   8%|▊         | 37/443 [00:31<01:11,  5.65it/s, Materializing param=model.layers.3.mlp.down_proj.weight]  
Loading weights:   8%|▊         | 37/443 [00:31<01:11,  5.65it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<02:13,  3.04it/s, Materializing param=model.layers.3.input_layernorm.weight] 
Loading weights:   8%|▊         | 36/443 [00:31<02:13,  3.04it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:   8%|▊         | 37/443 [00:31<02:13,  3.04it/s, Materializing param=model.layers.3.mlp.down_proj.weight]  
Loading weights:   8%|▊         | 37/443 [00:31<02:13,  3.04it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<01:09,  5.84it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<01:09,  5.84it/s, Materializing param=model.layers.3.input_layernorm.weight] 
Loading weights:   8%|▊         | 36/443 [00:31<01:09,  5.84it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:   8%|▊         | 37/443 [00:31<01:09,  5.84it/s, Materializing param=model.layers.3.mlp.down_proj.weight]  
Loading weights:   8%|▊         | 37/443 [00:31<01:09,  5.84it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<01:27,  4.64it/s, Materializing param=model.layers.3.input_layernorm.weight] 
Loading weights:   8%|▊         | 36/443 [00:31<01:27,  4.64it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:   8%|▊         | 37/443 [00:31<01:27,  4.64it/s, Materializing param=model.layers.3.mlp.down_proj.weight]  
Loading weights:   8%|▊         | 37/443 [00:31<01:27,  4.64it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<01:14,  5.44it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<01:14,  5.44it/s, Materializing param=model.layers.3.input_layernorm.weight] 
Loading weights:   8%|▊         | 36/443 [00:31<01:14,  5.44it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:   8%|▊         | 36/443 [00:31<02:17,  2.95it/s, Materializing param=model.layers.3.input_layernorm.weight] 
Loading weights:   8%|▊         | 36/443 [00:31<02:17,  2.95it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:   8%|▊         | 37/443 [00:31<02:17,  2.95it/s, Materializing param=model.layers.3.mlp.down_proj.weight]  
Loading weights:   8%|▊         | 37/443 [00:31<02:17,  2.95it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<01:17,  5.23it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
Loading weights:   8%|▊         | 36/443 [00:31<01:17,  5.23it/s, Materializing param=model.layers.3.input_layernorm.weight] 
Loading weights:   8%|▊         | 36/443 [00:31<01:17,  5.23it/s, Materializing param=model.layers.3.input_layernorm.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:17,  2.94it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   8%|▊         | 37/443 [00:33<01:17,  5.23it/s, Materializing param=model.layers.3.mlp.down_proj.weight]  
Loading weights:   9%|▊         | 38/443 [00:33<03:24,  1.98it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:45,  2.45it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:41,  2.50it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<03:14,  2.08it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<03:13,  2.09it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:17,  2.94it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   8%|▊         | 37/443 [00:33<01:17,  5.23it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:45,  2.45it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<03:24,  1.98it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:17,  2.94it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<03:13,  2.09it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<03:14,  2.08it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:45,  2.45it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<03:24,  1.98it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<03:13,  2.09it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<03:14,  2.08it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:48,  2.40it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:48,  2.40it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   8%|▊         | 37/443 [00:33<01:14,  5.44it/s, Materializing param=model.layers.3.mlp.down_proj.weight]  
Loading weights:   9%|▊         | 38/443 [00:33<02:48,  2.40it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   8%|▊         | 37/443 [00:33<01:14,  5.44it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:41,  2.50it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:45,  2.45it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:41,  2.50it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:45,  2.45it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▊         | 38/443 [00:33<02:45,  2.45it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:21,  1.55it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:17,  1.57it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:18,  1.56it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<03:34,  1.88it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:42,  1.43it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:43,  1.43it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:14,  1.59it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:17,  1.57it/s, Materializing param=model.layers.3.mlp.up_proj.weight]  
Loading weights:   9%|▉         | 39/443 [00:35<04:42,  1.43it/s, Materializing param=model.layers.3.mlp.up_proj.weight]  
Loading weights:   9%|▉         | 39/443 [00:35<04:21,  1.55it/s, Materializing param=model.layers.3.mlp.up_proj.weight]  
Loading weights:   9%|▉         | 39/443 [00:35<03:34,  1.88it/s, Materializing param=model.layers.3.mlp.up_proj.weight]  
Loading weights:   9%|▉         | 39/443 [00:35<04:18,  1.56it/s, Materializing param=model.layers.3.mlp.up_proj.weight]  
Loading weights:   9%|▉         | 39/443 [00:35<04:17,  1.57it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:42,  1.43it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:21,  1.55it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:43,  1.43it/s, Materializing param=model.layers.3.mlp.up_proj.weight]  
Loading weights:   9%|▉         | 39/443 [00:35<04:18,  1.56it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<03:34,  1.88it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:43,  1.43it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:56,  1.36it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:56,  1.36it/s, Materializing param=model.layers.3.mlp.up_proj.weight]  
Loading weights:   9%|▉         | 39/443 [00:35<04:56,  1.36it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 39/443 [00:35<04:14,  1.59it/s, Materializing param=model.layers.3.mlp.up_proj.weight]  
Loading weights:   9%|▉         | 39/443 [00:35<04:14,  1.59it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:50,  1.15it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 40/443 [00:37<06:28,  1.04it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 40/443 [00:37<06:28,  1.04it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:56,  1.13it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 40/443 [00:37<06:28,  1.04it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<06:15,  1.07it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 40/443 [00:37<06:13,  1.08it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:56,  1.13it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:52,  1.14it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 40/443 [00:37<04:59,  1.34it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:56,  1.13it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 41/443 [00:37<06:27,  1.04it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]        
Loading weights:   9%|▉         | 40/443 [00:37<06:13,  1.08it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<06:15,  1.07it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 41/443 [00:37<06:27,  1.04it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<06:13,  1.08it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<06:15,  1.07it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<04:59,  1.34it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:52,  1.14it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 41/443 [00:37<05:55,  1.13it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]        
Loading weights:   9%|▉         | 42/443 [00:37<06:27,  1.04it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:52,  1.14it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<04:59,  1.34it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:54,  1.14it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
Loading weights:   9%|▉         | 41/443 [00:37<06:12,  1.08it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]        
Loading weights:   9%|▉         | 42/443 [00:37<06:27,  1.04it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:50,  1.15it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 41/443 [00:37<06:12,  1.08it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights:   9%|▉         | 41/443 [00:37<05:55,  1.13it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:54,  1.14it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 41/443 [00:37<04:58,  1.34it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]        
Loading weights:   9%|▉         | 42/443 [00:37<06:11,  1.08it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 41/443 [00:37<05:51,  1.14it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]        
Loading weights:   9%|▉         | 41/443 [00:37<04:58,  1.34it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights:   9%|▉         | 41/443 [00:37<06:14,  1.07it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]        
Loading weights:   9%|▉         | 42/443 [00:37<06:11,  1.08it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 42/443 [00:37<05:55,  1.13it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 41/443 [00:37<05:51,  1.14it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:54,  1.14it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 41/443 [00:37<06:14,  1.07it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights:   9%|▉         | 42/443 [00:37<05:55,  1.13it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 42/443 [00:37<04:58,  1.34it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 42/443 [00:37<05:50,  1.14it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 42/443 [00:37<06:13,  1.07it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 42/443 [00:37<05:50,  1.14it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 42/443 [00:37<04:58,  1.34it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 41/443 [00:37<05:53,  1.14it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]        
Loading weights:   9%|▉         | 42/443 [00:37<06:13,  1.07it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 40/443 [00:37<05:50,  1.15it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
Loading weights:   9%|▉         | 41/443 [00:37<05:53,  1.14it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights:   9%|▉         | 42/443 [00:37<05:52,  1.14it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 42/443 [00:37<05:52,  1.14it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 41/443 [00:37<05:49,  1.15it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]        
Loading weights:   9%|▉         | 41/443 [00:37<05:49,  1.15it/s, Materializing param=model.layers.3.self_attn.k_norm.weight]
Loading weights:   9%|▉         | 42/443 [00:37<05:48,  1.15it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:   9%|▉         | 42/443 [00:37<05:48,  1.15it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:23,  1.97it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:23,  1.97it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:05,  2.15it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:23,  1.97it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:22,  1.98it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:20,  1.99it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:32,  1.88it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:22,  1.98it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:32,  1.88it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:32,  1.88it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:32,  1.88it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:22,  1.98it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:32,  1.88it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:32,  1.88it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:20,  1.99it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:20,  1.99it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:05,  2.15it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:05,  2.15it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:21,  1.98it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:21,  1.98it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:21,  1.98it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:36,  1.84it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:36,  1.84it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 43/443 [00:37<03:36,  1.84it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<03:36,  1.84it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|▉         | 44/443 [00:37<03:36,  1.84it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|█         | 45/443 [00:37<03:35,  1.84it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 45/443 [00:37<03:35,  1.84it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:53,  2.29it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:53,  2.29it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:53,  2.29it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|█         | 45/443 [00:37<02:53,  2.29it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 45/443 [00:37<02:53,  2.29it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<03:04,  2.17it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<03:04,  2.17it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|▉         | 44/443 [00:37<03:04,  2.17it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|█         | 45/443 [00:37<03:03,  2.17it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 45/443 [00:37<03:03,  2.17it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:47,  2.39it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:59,  2.22it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:47,  2.39it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:47,  2.39it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:59,  2.22it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:59,  2.22it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|█         | 45/443 [00:37<02:46,  2.39it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 45/443 [00:37<02:46,  2.39it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 45/443 [00:37<02:59,  2.22it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 45/443 [00:37<02:59,  2.22it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<03:07,  2.13it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<03:07,  2.13it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|▉         | 44/443 [00:37<03:07,  2.13it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|█         | 45/443 [00:37<03:07,  2.13it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 45/443 [00:37<03:07,  2.13it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:59,  2.22it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:59,  2.22it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|▉         | 44/443 [00:37<02:59,  2.22it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:05,  3.15it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:16,  2.90it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:05,  3.15it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:16,  2.90it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:11,  3.02it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:29,  2.66it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:11,  3.02it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:05,  3.15it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:16,  2.90it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:17,  2.89it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:29,  2.66it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:11,  3.02it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:29,  2.66it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:17,  2.89it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:17,  2.89it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:11,  3.02it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:11,  3.02it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 45/443 [00:37<02:37,  2.53it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|█         | 45/443 [00:37<02:37,  2.53it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 45/443 [00:37<02:37,  2.53it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:36,  2.53it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:11,  3.02it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<02:36,  2.53it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<03:14,  2.05it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
Loading weights:  10%|▉         | 44/443 [00:37<03:14,  2.05it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|▉         | 44/443 [00:37<03:14,  2.05it/s, Materializing param=model.layers.3.self_attn.q_norm.weight]
Loading weights:  10%|█         | 45/443 [00:37<03:14,  2.05it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 45/443 [00:37<03:14,  2.05it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<03:13,  2.05it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  10%|█         | 46/443 [00:37<03:13,  2.05it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  11%|█         | 47/443 [00:37<01:54,  3.45it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  11%|█         | 47/443 [00:37<01:58,  3.33it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  11%|█         | 47/443 [00:37<01:54,  3.45it/s, Materializing param=model.layers.4.input_layernorm.weight] 
Loading weights:  11%|█         | 47/443 [00:37<02:12,  2.99it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  11%|█         | 47/443 [00:37<01:54,  3.45it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|█         | 47/443 [00:37<02:12,  2.99it/s, Materializing param=model.layers.4.input_layernorm.weight] 
Loading weights:  11%|█         | 47/443 [00:37<01:58,  3.33it/s, Materializing param=model.layers.4.input_layernorm.weight] 
Loading weights:  11%|█         | 47/443 [00:37<02:12,  2.99it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|█         | 47/443 [00:37<01:58,  3.33it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|█         | 47/443 [00:37<03:13,  2.05it/s, Materializing param=model.layers.4.input_layernorm.weight] 
Loading weights:  11%|█         | 47/443 [00:37<01:51,  3.56it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  11%|█         | 47/443 [00:37<03:13,  2.05it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|█         | 47/443 [00:37<01:47,  3.67it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  11%|█         | 47/443 [00:37<01:51,  3.56it/s, Materializing param=model.layers.4.input_layernorm.weight] 
Loading weights:  11%|█         | 47/443 [00:37<01:54,  3.45it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  11%|█         | 47/443 [00:37<01:51,  3.56it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|█         | 48/443 [00:37<02:12,  2.99it/s, Materializing param=model.layers.4.mlp.down_proj.weight]  
Loading weights:  11%|█         | 47/443 [00:37<01:47,  3.67it/s, Materializing param=model.layers.4.input_layernorm.weight] 
Loading weights:  11%|█         | 47/443 [00:37<01:59,  3.31it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
Loading weights:  11%|█         | 48/443 [00:37<03:12,  2.05it/s, Materializing param=model.layers.4.mlp.down_proj.weight]  
Loading weights:  11%|█         | 47/443 [00:37<01:47,  3.67it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|█         | 48/443 [00:37<03:12,  2.05it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 48/443 [00:37<02:12,  2.99it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 47/443 [00:37<01:59,  3.31it/s, Materializing param=model.layers.4.input_layernorm.weight] 
Loading weights:  11%|█         | 48/443 [00:37<01:50,  3.56it/s, Materializing param=model.layers.4.mlp.down_proj.weight]  
Loading weights:  11%|█         | 47/443 [00:37<01:59,  3.31it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|█         | 48/443 [00:37<01:50,  3.56it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 48/443 [00:37<01:47,  3.67it/s, Materializing param=model.layers.4.mlp.down_proj.weight]  
Loading weights:  11%|█         | 48/443 [00:37<01:47,  3.67it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 47/443 [00:37<01:54,  3.45it/s, Materializing param=model.layers.4.input_layernorm.weight] 
Loading weights:  11%|█         | 48/443 [00:37<01:59,  3.31it/s, Materializing param=model.layers.4.mlp.down_proj.weight]  
Loading weights:  11%|█         | 48/443 [00:37<01:59,  3.31it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 47/443 [00:37<01:54,  3.45it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|█         | 48/443 [00:37<01:54,  3.45it/s, Materializing param=model.layers.4.mlp.down_proj.weight]  
Loading weights:  11%|█         | 48/443 [00:37<01:54,  3.45it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 48/443 [00:39<04:20,  1.51it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|█         | 48/443 [00:39<04:20,  1.51it/s, Materializing param=model.layers.4.mlp.down_proj.weight]  
Loading weights:  11%|█         | 48/443 [00:39<04:20,  1.51it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<02:54,  2.25it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:25,  1.92it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:30,  1.87it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:23,  1.94it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 48/443 [00:39<04:17,  1.53it/s, Materializing param=model.layers.4.input_layernorm.weight]
Loading weights:  11%|█         | 49/443 [00:39<02:54,  2.25it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:25,  1.92it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:34,  1.84it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:23,  1.94it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<04:20,  1.51it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<02:54,  2.25it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:25,  1.92it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:31,  1.86it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:23,  1.94it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<04:20,  1.51it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:34,  1.84it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 48/443 [00:39<04:17,  1.53it/s, Materializing param=model.layers.4.mlp.down_proj.weight]  
Loading weights:  11%|█         | 48/443 [00:39<04:17,  1.53it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:31,  1.86it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:34,  1.84it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:31,  1.86it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:30,  1.87it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<04:16,  1.53it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<03:30,  1.87it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█         | 49/443 [00:39<04:16,  1.53it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:25,  1.21it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:37,  1.16it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:45,  1.14it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:27,  1.20it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:48,  1.13it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:25,  1.21it/s, Materializing param=model.layers.4.mlp.up_proj.weight]  
Loading weights:  11%|█▏        | 50/443 [00:42<05:27,  1.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]  
Loading weights:  11%|█▏        | 50/443 [00:42<05:37,  1.16it/s, Materializing param=model.layers.4.mlp.up_proj.weight]  
Loading weights:  11%|█▏        | 50/443 [00:42<05:48,  1.13it/s, Materializing param=model.layers.4.mlp.up_proj.weight]  
Loading weights:  11%|█▏        | 50/443 [00:42<05:27,  1.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:25,  1.21it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:37,  1.16it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:32,  1.18it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:48,  1.13it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:35,  1.17it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:32,  1.18it/s, Materializing param=model.layers.4.mlp.up_proj.weight]  
Loading weights:  11%|█▏        | 50/443 [00:42<05:32,  1.18it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:35,  1.17it/s, Materializing param=model.layers.4.mlp.up_proj.weight]  
Loading weights:  11%|█▏        | 50/443 [00:42<05:35,  1.17it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<04:30,  1.45it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:45,  1.14it/s, Materializing param=model.layers.4.mlp.up_proj.weight]  
Loading weights:  11%|█▏        | 50/443 [00:42<04:30,  1.45it/s, Materializing param=model.layers.4.mlp.up_proj.weight]  
Loading weights:  11%|█▏        | 50/443 [00:42<04:30,  1.45it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  11%|█▏        | 50/443 [00:42<05:45,  1.14it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<04:30,  1.45it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<04:30,  1.45it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 52/443 [00:42<04:29,  1.45it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]        
Loading weights:  12%|█▏        | 52/443 [00:42<04:29,  1.45it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<04:28,  1.45it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<04:28,  1.45it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<04:28,  1.45it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<04:28,  1.45it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:42<02:29,  2.59it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:42<02:29,  2.59it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 55/443 [00:42<02:29,  2.59it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:18,  1.23it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:18,  1.23it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:18,  1.23it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 52/443 [00:42<05:17,  1.23it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]        
Loading weights:  12%|█▏        | 52/443 [00:42<05:17,  1.23it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:17,  1.23it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:17,  1.23it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:28,  1.19it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:28,  1.19it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:28,  1.19it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 52/443 [00:42<05:27,  1.19it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]        
Loading weights:  12%|█▏        | 52/443 [00:42<05:27,  1.19it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:26,  1.19it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:26,  1.19it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:14,  1.25it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:14,  1.25it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:14,  1.25it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 52/443 [00:42<05:13,  1.25it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]        
Loading weights:  12%|█▏        | 52/443 [00:42<05:13,  1.25it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:12,  1.25it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:12,  1.25it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:32,  1.18it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:32,  1.18it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:32,  1.18it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 52/443 [00:42<05:31,  1.18it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]        
Loading weights:  12%|█▏        | 52/443 [00:42<05:31,  1.18it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:30,  1.18it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:30,  1.18it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:16,  1.23it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:16,  1.23it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  13%|█▎        | 56/443 [00:42<02:27,  2.62it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:18,  1.23it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:18,  1.23it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:18,  1.23it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  13%|█▎        | 56/443 [00:42<02:27,  2.62it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 56/443 [00:42<02:27,  2.62it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:42<02:27,  2.62it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:42<02:27,  2.62it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  12%|█▏        | 52/443 [00:42<05:17,  1.23it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]        
Loading weights:  12%|█▏        | 52/443 [00:42<05:17,  1.23it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:17,  1.23it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:17,  1.23it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:11,  1.25it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:11,  1.25it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:25,  1.19it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:25,  1.19it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:25,  1.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:25,  1.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:25,  1.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 52/443 [00:42<05:24,  1.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]        
Loading weights:  12%|█▏        | 52/443 [00:42<05:24,  1.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:24,  1.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:24,  1.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:29,  1.18it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:29,  1.18it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:27,  1.20it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:27,  1.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 51/443 [00:42<05:27,  1.20it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
Loading weights:  12%|█▏        | 52/443 [00:42<05:26,  1.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]        
Loading weights:  12%|█▏        | 52/443 [00:42<05:26,  1.20it/s, Materializing param=model.layers.4.self_attn.k_norm.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:26,  1.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 53/443 [00:42<05:26,  1.20it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:16,  1.23it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:16,  1.23it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  13%|█▎        | 58/443 [00:42<02:26,  2.62it/s, Materializing param=model.layers.5.input_layernorm.weight] 
Loading weights:  13%|█▎        | 58/443 [00:42<02:26,  2.62it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  13%|█▎        | 59/443 [00:42<02:26,  2.62it/s, Materializing param=model.layers.5.mlp.down_proj.weight]  
Loading weights:  13%|█▎        | 59/443 [00:42<02:26,  2.62it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:25,  1.20it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:42<05:25,  1.20it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:33,  2.53it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:33,  2.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:33,  2.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 54/443 [00:43<02:55,  2.22it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:43<02:55,  2.22it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 54/443 [00:43<02:55,  2.22it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:54,  2.22it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:54,  2.22it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:54,  2.22it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:54,  2.22it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:35,  2.49it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:35,  2.49it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:35,  2.49it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:36,  2.48it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:36,  2.48it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:36,  2.48it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:36,  2.48it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:36,  2.48it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:32,  2.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:32,  2.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:55,  3.35it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:55,  3.35it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:55,  3.35it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:33,  2.53it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:33,  2.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:33,  2.53it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:32,  2.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:32,  2.53it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 58/443 [00:43<01:54,  3.35it/s, Materializing param=model.layers.5.input_layernorm.weight] 
Loading weights:  13%|█▎        | 58/443 [00:43<01:54,  3.35it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  13%|█▎        | 59/443 [00:43<01:54,  3.35it/s, Materializing param=model.layers.5.mlp.down_proj.weight]  
Loading weights:  13%|█▎        | 59/443 [00:43<01:54,  3.35it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:34,  2.51it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:34,  2.51it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:34,  2.51it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:33,  2.51it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:33,  2.51it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:37,  2.47it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:37,  2.47it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  12%|█▏        | 55/443 [00:43<02:37,  2.47it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:36,  2.47it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:36,  2.47it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:56,  3.33it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:56,  3.33it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:56,  3.33it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:53,  3.40it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:53,  3.40it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:53,  3.40it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 58/443 [00:43<01:55,  3.33it/s, Materializing param=model.layers.5.input_layernorm.weight] 
Loading weights:  13%|█▎        | 58/443 [00:43<01:55,  3.33it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  13%|█▎        | 59/443 [00:43<01:55,  3.33it/s, Materializing param=model.layers.5.mlp.down_proj.weight]  
Loading weights:  13%|█▎        | 59/443 [00:43<01:55,  3.33it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:57,  3.27it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:57,  3.27it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:57,  3.27it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 58/443 [00:43<01:53,  3.40it/s, Materializing param=model.layers.5.input_layernorm.weight] 
Loading weights:  13%|█▎        | 58/443 [00:43<01:53,  3.40it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  13%|█▎        | 59/443 [00:43<01:52,  3.40it/s, Materializing param=model.layers.5.mlp.down_proj.weight]  
Loading weights:  13%|█▎        | 59/443 [00:43<01:52,  3.40it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:59,  3.22it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:59,  3.22it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<01:59,  3.22it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 58/443 [00:43<01:57,  3.27it/s, Materializing param=model.layers.5.input_layernorm.weight] 
Loading weights:  13%|█▎        | 58/443 [00:43<01:57,  3.27it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  13%|█▎        | 59/443 [00:43<01:57,  3.27it/s, Materializing param=model.layers.5.mlp.down_proj.weight]  
Loading weights:  13%|█▎        | 59/443 [00:43<01:57,  3.27it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:23,  2.70it/s, Materializing param=model.layers.4.self_attn.q_norm.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:23,  2.70it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 56/443 [00:43<02:23,  2.70it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<02:23,  2.70it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<02:23,  2.70it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<02:03,  3.13it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<02:03,  3.13it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 57/443 [00:43<02:03,  3.13it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
Loading weights:  13%|█▎        | 58/443 [00:43<01:59,  3.22it/s, Materializing param=model.layers.5.input_layernorm.weight] 
Loading weights:  13%|█▎        | 58/443 [00:43<01:59,  3.22it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  13%|█▎        | 59/443 [00:43<01:59,  3.22it/s, Materializing param=model.layers.5.mlp.down_proj.weight]  
Loading weights:  13%|█▎        | 59/443 [00:43<01:59,  3.22it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  13%|█▎        | 58/443 [00:43<02:22,  2.70it/s, Materializing param=model.layers.5.input_layernorm.weight] 
Loading weights:  13%|█▎        | 58/443 [00:43<02:22,  2.70it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  13%|█▎        | 59/443 [00:43<02:22,  2.70it/s, Materializing param=model.layers.5.mlp.down_proj.weight]  
Loading weights:  13%|█▎        | 59/443 [00:43<02:22,  2.70it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  13%|█▎        | 58/443 [00:43<02:02,  3.13it/s, Materializing param=model.layers.5.input_layernorm.weight] 
Loading weights:  13%|█▎        | 58/443 [00:43<02:02,  3.13it/s, Materializing param=model.layers.5.input_layernorm.weight]
Loading weights:  13%|█▎        | 59/443 [00:43<02:02,  3.13it/s, Materializing param=model.layers.5.mlp.down_proj.weight]  
Loading weights:  13%|█▎        | 59/443 [00:43<02:02,  3.13it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:12,  2.89it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:14,  2.84it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:12,  2.89it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:14,  2.84it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:31,  2.53it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:12,  2.89it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:14,  2.84it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:10,  2.93it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:14,  2.85it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:31,  2.53it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:15,  2.83it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:14,  2.85it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:31,  2.53it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:20,  2.72it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:14,  2.85it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:15,  2.83it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:15,  2.82it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:20,  2.72it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:15,  2.83it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:44<02:31,  2.53it/s, Materializing param=model.layers.5.mlp.up_proj.weight]  
Loading weights:  14%|█▎        | 60/443 [00:44<02:20,  2.72it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:10,  2.93it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:15,  2.82it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:44<02:31,  2.53it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:15,  2.82it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:44<02:15,  2.83it/s, Materializing param=model.layers.5.mlp.up_proj.weight]  
Loading weights:  14%|█▍        | 61/443 [00:44<02:15,  2.83it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▎        | 60/443 [00:44<02:10,  2.93it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:24,  2.65it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:24,  2.65it/s, Materializing param=model.layers.5.mlp.up_proj.weight]  
Loading weights:  14%|█▍        | 61/443 [00:45<02:24,  2.65it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:29,  2.55it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:29,  2.55it/s, Materializing param=model.layers.5.mlp.up_proj.weight]  
Loading weights:  14%|█▍        | 61/443 [00:45<02:29,  2.55it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:26,  2.61it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:26,  2.61it/s, Materializing param=model.layers.5.mlp.up_proj.weight]  
Loading weights:  14%|█▍        | 61/443 [00:45<02:26,  2.61it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:27,  2.58it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:27,  2.58it/s, Materializing param=model.layers.5.mlp.up_proj.weight]  
Loading weights:  14%|█▍        | 61/443 [00:45<02:27,  2.58it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:24,  2.65it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:24,  2.65it/s, Materializing param=model.layers.5.mlp.up_proj.weight]  
Loading weights:  14%|█▍        | 61/443 [00:45<02:24,  2.65it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:26,  2.61it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
Loading weights:  14%|█▍        | 61/443 [00:45<02:26,  2.61it/s, Materializing param=model.layers.5.mlp.up_proj.weight]  
Loading weights:  14%|█▍        | 61/443 [00:45<02:26,  2.61it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:40,  1.73it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:24,  1.86it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:24,  1.86it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:24,  1.86it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:40,  1.73it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:40,  1.73it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:21,  1.89it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:35,  1.77it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:21,  1.89it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:40,  1.73it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]        
Loading weights:  14%|█▍        | 63/443 [00:46<03:40,  1.73it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:40,  1.73it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:21,  1.89it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:41,  1.72it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:44,  1.70it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:36,  1.76it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:39,  1.73it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:40,  1.73it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:39,  1.73it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:41,  1.72it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:36,  1.76it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:44,  1.70it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:41,  1.72it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:40,  1.73it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:36,  1.76it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:20,  1.89it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]        
Loading weights:  14%|█▍        | 62/443 [00:46<03:44,  1.70it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:20,  1.89it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:35,  1.76it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]        
Loading weights:  14%|█▍        | 63/443 [00:46<03:39,  1.73it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]        
Loading weights:  14%|█▍        | 62/443 [00:46<03:35,  1.77it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:20,  1.89it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:35,  1.76it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:39,  1.73it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:40,  1.72it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]        
Loading weights:  14%|█▍        | 63/443 [00:46<03:43,  1.70it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]        
Loading weights:  14%|█▍        | 64/443 [00:46<03:20,  1.89it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:43,  1.70it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:40,  1.72it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights:  14%|█▍        | 62/443 [00:46<03:35,  1.77it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:35,  1.76it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:39,  1.73it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:35,  1.76it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:39,  1.73it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:43,  1.70it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:39,  1.72it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:43,  1.70it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:39,  1.72it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:34,  1.77it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]        
Loading weights:  14%|█▍        | 63/443 [00:46<03:34,  1.77it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:33,  1.77it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:33,  1.77it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:15,  2.80it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:15,  2.80it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:14,  2.80it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:15,  2.80it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:14,  2.80it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:12,  2.85it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:12,  2.85it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:14,  2.80it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:12,  2.85it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:04,  2.06it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:12,  2.85it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:04,  2.06it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]        
Loading weights:  15%|█▍        | 65/443 [00:46<02:12,  2.85it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  14%|█▍        | 63/443 [00:46<03:04,  2.06it/s, Materializing param=model.layers.5.self_attn.k_norm.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:12,  2.85it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:03,  2.06it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  14%|█▍        | 64/443 [00:46<03:03,  2.06it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<03:03,  2.06it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<03:03,  2.06it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:12,  2.84it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:11,  2.88it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:12,  2.84it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:10,  2.89it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:12,  2.84it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:10,  2.89it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:11,  2.88it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:11,  2.88it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 65/443 [00:46<02:10,  2.89it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:10,  2.88it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:10,  2.88it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▌        | 67/443 [00:46<02:10,  2.88it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 67/443 [00:46<02:10,  2.88it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:03,  3.05it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:03,  3.05it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:03,  3.05it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:05,  3.02it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:05,  3.02it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:05,  3.02it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▌        | 67/443 [00:46<02:04,  3.02it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 67/443 [00:46<02:04,  3.02it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:03,  3.05it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:03,  3.05it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:03,  3.05it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:04,  3.03it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:04,  3.03it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:04,  3.03it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:04,  3.03it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:04,  3.03it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:04,  3.03it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▌        | 67/443 [00:46<02:04,  3.03it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 67/443 [00:46<02:04,  3.03it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:04,  3.02it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:04,  3.02it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▍        | 66/443 [00:46<02:04,  3.02it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▌        | 67/443 [00:46<02:04,  3.02it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 67/443 [00:46<02:04,  3.02it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:35,  3.94it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:35,  3.94it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:35,  3.94it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 67/443 [00:47<01:52,  3.34it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▌        | 67/443 [00:47<01:52,  3.34it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 67/443 [00:47<01:52,  3.34it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:52,  3.34it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:52,  3.34it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:38,  3.82it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:38,  3.82it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:38,  3.82it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 67/443 [00:47<01:54,  3.28it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▌        | 67/443 [00:47<01:54,  3.28it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 67/443 [00:47<01:54,  3.28it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:54,  3.28it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:54,  3.28it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:33,  4.00it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:33,  4.00it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:33,  4.00it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 67/443 [00:47<01:52,  3.33it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▌        | 67/443 [00:47<01:52,  3.33it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 67/443 [00:47<01:52,  3.33it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:52,  3.33it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:52,  3.33it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:34,  3.98it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:34,  3.98it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<01:34,  3.98it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  16%|█▌        | 69/443 [00:47<01:54,  3.28it/s, Materializing param=model.layers.6.input_layernorm.weight] 
Loading weights:  16%|█▌        | 69/443 [00:47<01:54,  3.28it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  16%|█▌        | 69/443 [00:47<01:37,  3.82it/s, Materializing param=model.layers.6.input_layernorm.weight] 
Loading weights:  16%|█▌        | 69/443 [00:47<01:27,  4.29it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  16%|█▌        | 69/443 [00:47<01:37,  3.82it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  16%|█▌        | 69/443 [00:47<01:52,  3.34it/s, Materializing param=model.layers.6.input_layernorm.weight] 
Loading weights:  16%|█▌        | 69/443 [00:47<01:27,  4.29it/s, Materializing param=model.layers.6.input_layernorm.weight] 
Loading weights:  16%|█▌        | 69/443 [00:47<01:52,  3.34it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  16%|█▌        | 69/443 [00:47<01:27,  4.29it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  16%|█▌        | 70/443 [00:47<01:37,  3.82it/s, Materializing param=model.layers.6.mlp.down_proj.weight]  
Loading weights:  16%|█▌        | 70/443 [00:47<01:53,  3.28it/s, Materializing param=model.layers.6.mlp.down_proj.weight]  
Loading weights:  16%|█▌        | 70/443 [00:47<01:37,  3.82it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 70/443 [00:47<01:53,  3.28it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 70/443 [00:47<01:26,  4.29it/s, Materializing param=model.layers.6.mlp.down_proj.weight]  
Loading weights:  16%|█▌        | 69/443 [00:47<01:33,  4.00it/s, Materializing param=model.layers.6.input_layernorm.weight] 
Loading weights:  16%|█▌        | 70/443 [00:47<01:51,  3.34it/s, Materializing param=model.layers.6.mlp.down_proj.weight]  
Loading weights:  16%|█▌        | 70/443 [00:47<01:26,  4.29it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 69/443 [00:47<01:52,  3.33it/s, Materializing param=model.layers.6.input_layernorm.weight] 
Loading weights:  16%|█▌        | 69/443 [00:47<01:33,  4.00it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  16%|█▌        | 69/443 [00:47<01:52,  3.33it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  16%|█▌        | 70/443 [00:47<01:51,  3.34it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 70/443 [00:47<01:33,  4.00it/s, Materializing param=model.layers.6.mlp.down_proj.weight]  
Loading weights:  16%|█▌        | 70/443 [00:47<01:33,  4.00it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 69/443 [00:47<01:33,  3.98it/s, Materializing param=model.layers.6.input_layernorm.weight] 
Loading weights:  16%|█▌        | 69/443 [00:47<01:33,  3.98it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  16%|█▌        | 70/443 [00:47<01:33,  3.98it/s, Materializing param=model.layers.6.mlp.down_proj.weight]  
Loading weights:  16%|█▌        | 70/443 [00:47<01:33,  3.98it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:47<02:15,  2.77it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
Loading weights:  15%|█▍        | 66/443 [00:47<02:15,  2.77it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▍        | 66/443 [00:47<02:15,  2.77it/s, Materializing param=model.layers.5.self_attn.q_norm.weight]
Loading weights:  15%|█▌        | 67/443 [00:47<02:15,  2.77it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 67/443 [00:47<02:15,  2.77it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<02:15,  2.77it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  15%|█▌        | 68/443 [00:47<02:15,  2.77it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
Loading weights:  16%|█▌        | 69/443 [00:47<02:14,  2.77it/s, Materializing param=model.layers.6.input_layernorm.weight] 
Loading weights:  16%|█▌        | 69/443 [00:47<02:14,  2.77it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  16%|█▌        | 70/443 [00:47<02:14,  2.77it/s, Materializing param=model.layers.6.mlp.down_proj.weight]  
Loading weights:  16%|█▌        | 70/443 [00:47<02:14,  2.77it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:28,  2.50it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:15,  2.74it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 70/443 [00:48<02:30,  2.48it/s, Materializing param=model.layers.6.input_layernorm.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:03,  3.00it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:15,  2.74it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:17,  2.71it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:12,  2.81it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:15,  2.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 70/443 [00:48<02:30,  2.48it/s, Materializing param=model.layers.6.mlp.down_proj.weight]  
Loading weights:  16%|█▌        | 71/443 [00:48<02:28,  2.50it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:15,  2.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:15,  2.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:28,  2.50it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:17,  2.71it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 70/443 [00:48<02:30,  2.48it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:15,  2.74it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:12,  2.81it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:12,  2.81it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:17,  2.71it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:48<02:11,  2.81it/s, Materializing param=model.layers.6.mlp.up_proj.weight]  
Loading weights:  16%|█▌        | 71/443 [00:48<02:03,  3.00it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:48<02:11,  2.81it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:30,  2.48it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:30,  2.48it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:48<02:03,  3.00it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:49<02:19,  2.66it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:49<02:29,  2.49it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:49<02:19,  2.66it/s, Materializing param=model.layers.6.mlp.up_proj.weight]  
Loading weights:  16%|█▋        | 72/443 [00:49<02:29,  2.49it/s, Materializing param=model.layers.6.mlp.up_proj.weight]  
Loading weights:  16%|█▋        | 72/443 [00:49<02:19,  2.66it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:49<02:18,  2.68it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:49<02:29,  2.49it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:49<02:18,  2.68it/s, Materializing param=model.layers.6.mlp.up_proj.weight]  
Loading weights:  16%|█▋        | 72/443 [00:49<02:18,  2.68it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:49<02:18,  2.68it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:49<02:18,  2.68it/s, Materializing param=model.layers.6.mlp.up_proj.weight]  
Loading weights:  16%|█▋        | 72/443 [00:49<02:18,  2.68it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<03:03,  2.02it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<02:52,  2.15it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<02:52,  2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<02:52,  2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  17%|█▋        | 74/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]        
Loading weights:  17%|█▋        | 74/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<03:03,  2.02it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<03:03,  2.02it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<02:40,  2.31it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<02:52,  2.15it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<02:52,  2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  17%|█▋        | 74/443 [00:50<03:02,  2.02it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]        
Loading weights:  16%|█▋        | 73/443 [00:50<02:40,  2.31it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<02:52,  2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  17%|█▋        | 74/443 [00:50<03:02,  2.02it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<02:40,  2.31it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<03:02,  2.02it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 74/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]        
Loading weights:  17%|█▋        | 75/443 [00:50<03:02,  2.02it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 74/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights:  17%|█▋        | 74/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]        
Loading weights:  17%|█▋        | 74/443 [00:50<02:39,  2.31it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]        
Loading weights:  17%|█▋        | 75/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 74/443 [00:50<02:39,  2.31it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights:  17%|█▋        | 74/443 [00:50<02:51,  2.15it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<02:50,  2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<02:39,  2.31it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<02:50,  2.15it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<02:39,  2.31it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:45,  3.47it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:41,  3.63it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:45,  3.47it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:41,  3.63it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:45,  3.47it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:41,  3.63it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:42,  3.58it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:44,  3.51it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:44,  3.51it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:42,  3.58it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:44,  3.51it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:42,  3.58it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:45,  3.47it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:45,  3.47it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  16%|█▌        | 71/443 [00:50<03:20,  1.86it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:50<03:20,  1.86it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▌        | 71/443 [00:50<03:20,  1.86it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:50<03:19,  1.86it/s, Materializing param=model.layers.6.mlp.up_proj.weight]  
Loading weights:  16%|█▋        | 72/443 [00:50<03:19,  1.86it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<03:19,  1.86it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<03:19,  1.86it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  17%|█▋        | 74/443 [00:50<03:18,  1.86it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]        
Loading weights:  17%|█▋        | 74/443 [00:50<03:18,  1.86it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<03:18,  1.86it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<03:18,  1.86it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:50<03:16,  1.89it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:53,  3.22it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:50<03:16,  1.89it/s, Materializing param=model.layers.6.mlp.up_proj.weight]  
Loading weights:  16%|█▋        | 72/443 [00:50<03:16,  1.89it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<03:15,  1.89it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<03:15,  1.89it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  17%|█▋        | 74/443 [00:50<03:15,  1.89it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]        
Loading weights:  17%|█▋        | 74/443 [00:50<03:15,  1.89it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<03:14,  1.89it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<03:14,  1.89it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:53,  3.22it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<01:53,  3.22it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:53,  3.22it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:53,  3.22it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:53,  3.22it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:53,  3.22it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:39,  3.69it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:39,  3.69it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:39,  3.69it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:38,  3.69it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:42,  3.58it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:38,  3.69it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:42,  3.58it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:25,  4.28it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:25,  4.28it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:25,  4.28it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:41,  3.59it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:41,  3.59it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:41,  3.59it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:41,  3.59it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:41,  3.59it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<03:17,  1.86it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<03:17,  1.86it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<03:14,  1.89it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<03:14,  1.89it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:50<03:15,  1.90it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
Loading weights:  16%|█▋        | 72/443 [00:50<03:15,  1.90it/s, Materializing param=model.layers.6.mlp.up_proj.weight]  
Loading weights:  16%|█▋        | 72/443 [00:50<03:15,  1.90it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<03:14,  1.90it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  16%|█▋        | 73/443 [00:50<03:14,  1.90it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
Loading weights:  17%|█▋        | 74/443 [00:50<03:14,  1.90it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]        
Loading weights:  17%|█▋        | 74/443 [00:50<03:14,  1.90it/s, Materializing param=model.layers.6.self_attn.k_norm.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<03:13,  1.90it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 75/443 [00:50<03:13,  1.90it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<03:13,  1.90it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 76/443 [00:50<03:13,  1.90it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:43,  3.52it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:43,  3.52it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:43,  3.52it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:43,  3.52it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:43,  3.52it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:38,  3.70it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:38,  3.70it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<01:38,  3.70it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:38,  3.70it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:38,  3.70it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<03:12,  1.90it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  17%|█▋        | 77/443 [00:50<03:12,  1.90it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:46,  3.41it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:46,  3.41it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:46,  3.41it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:31,  3.98it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:31,  3.98it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:27,  4.14it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:31,  3.98it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:37,  3.75it/s, Materializing param=model.layers.6.self_attn.q_norm.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:25,  4.27it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:46,  3.41it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:26,  4.19it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:27,  4.14it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:46,  3.41it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:27,  4.14it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:25,  4.27it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:26,  4.19it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:25,  4.27it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:26,  4.19it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:33,  3.90it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:33,  3.90it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:37,  3.75it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:33,  3.90it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 78/443 [00:50<01:37,  3.75it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:36,  3.75it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:36,  3.75it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:29,  4.07it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:29,  4.07it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 79/443 [00:50<01:29,  4.07it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:18,  4.65it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:31,  3.98it/s, Materializing param=model.layers.7.input_layernorm.weight] 
Loading weights:  18%|█▊        | 80/443 [00:50<01:19,  4.59it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:24,  4.27it/s, Materializing param=model.layers.7.input_layernorm.weight] 
Loading weights:  18%|█▊        | 80/443 [00:50<01:31,  3.98it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:19,  4.59it/s, Materializing param=model.layers.7.input_layernorm.weight] 
Loading weights:  18%|█▊        | 80/443 [00:50<01:18,  4.65it/s, Materializing param=model.layers.7.input_layernorm.weight] 
Loading weights:  18%|█▊        | 80/443 [00:50<01:19,  4.59it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:18,  4.65it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|█▊        | 81/443 [00:50<01:17,  4.65it/s, Materializing param=model.layers.7.mlp.down_proj.weight]  
Loading weights:  18%|█▊        | 81/443 [00:50<01:17,  4.65it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:22,  4.41it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:22,  4.38it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:24,  4.27it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:19,  4.58it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:22,  4.41it/s, Materializing param=model.layers.7.input_layernorm.weight] 
Loading weights:  18%|█▊        | 80/443 [00:50<01:22,  4.38it/s, Materializing param=model.layers.7.input_layernorm.weight] 
Loading weights:  18%|█▊        | 80/443 [00:50<01:22,  4.41it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:20,  4.48it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:19,  4.58it/s, Materializing param=model.layers.7.input_layernorm.weight] 
Loading weights:  18%|█▊        | 80/443 [00:50<01:22,  4.38it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:19,  4.58it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|█▊        | 81/443 [00:50<01:08,  5.27it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|█▊        | 81/443 [00:50<01:22,  4.41it/s, Materializing param=model.layers.7.mlp.down_proj.weight]  
Loading weights:  18%|█▊        | 81/443 [00:50<01:08,  5.27it/s, Materializing param=model.layers.7.mlp.down_proj.weight]  
Loading weights:  18%|█▊        | 81/443 [00:50<01:22,  4.38it/s, Materializing param=model.layers.7.mlp.down_proj.weight]  
Loading weights:  18%|█▊        | 81/443 [00:50<01:22,  4.41it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  18%|█▊        | 81/443 [00:50<01:08,  5.27it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  18%|█▊        | 81/443 [00:50<01:19,  4.58it/s, Materializing param=model.layers.7.mlp.down_proj.weight]  
Loading weights:  18%|█▊        | 81/443 [00:50<01:22,  4.38it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  18%|█▊        | 81/443 [00:50<01:19,  4.58it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  18%|█▊        | 80/443 [00:50<01:20,  4.48it/s, Materializing param=model.layers.7.input_layernorm.weight] 
Loading weights:  18%|█▊        | 80/443 [00:50<01:20,  4.48it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|█▊        | 81/443 [00:50<01:20,  4.48it/s, Materializing param=model.layers.7.mlp.down_proj.weight]  
Loading weights:  18%|█▊        | 81/443 [00:50<01:20,  4.48it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  18%|█▊        | 81/443 [00:52<02:37,  2.30it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|█▊        | 81/443 [00:52<03:24,  1.77it/s, Materializing param=model.layers.7.input_layernorm.weight]
Loading weights:  18%|█▊        | 81/443 [00:52<02:37,  2.30it/s, Materializing param=model.layers.7.mlp.down_proj.weight]  
Loading weights:  18%|█▊        | 81/443 [00:52<02:37,  2.30it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  18%|█▊        | 81/443 [00:52<03:24,  1.77it/s, Materializing param=model.layers.7.mlp.down_proj.weight]  
Loading weights:  18%|█▊        | 81/443 [00:52<03:24,  1.77it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<02:28,  2.44it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<03:07,  1.93it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<02:45,  2.18it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<01:08,  5.27it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<03:07,  1.93it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<03:10,  1.90it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<03:05,  1.95it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<03:07,  1.93it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<03:23,  1.77it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<01:08,  5.27it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<02:45,  2.18it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<02:37,  2.30it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<02:45,  2.18it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<02:37,  2.30it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<03:10,  1.90it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<03:05,  1.95it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<03:23,  1.77it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<03:10,  1.90it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<02:28,  2.44it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<03:05,  1.95it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 82/443 [00:52<02:28,  2.44it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<04:00,  1.50it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<04:01,  1.49it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<03:50,  1.56it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<04:00,  1.50it/s, Materializing param=model.layers.7.mlp.up_proj.weight]  
Loading weights:  19%|█▊        | 83/443 [00:54<04:01,  1.49it/s, Materializing param=model.layers.7.mlp.up_proj.weight]  
Loading weights:  19%|█▊        | 83/443 [00:54<04:34,  1.31it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<04:36,  1.30it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<03:33,  1.68it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<04:39,  1.29it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<04:00,  1.50it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<04:01,  1.49it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<04:36,  1.30it/s, Materializing param=model.layers.7.mlp.up_proj.weight]  
Loading weights:  19%|█▊        | 83/443 [00:54<03:50,  1.56it/s, Materializing param=model.layers.7.mlp.up_proj.weight]  
Loading weights:  19%|█▊        | 83/443 [00:54<04:36,  1.30it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<04:34,  1.31it/s, Materializing param=model.layers.7.mlp.up_proj.weight]  
Loading weights:  19%|█▊        | 83/443 [00:54<04:39,  1.29it/s, Materializing param=model.layers.7.mlp.up_proj.weight]  
Loading weights:  19%|█▊        | 83/443 [00:54<03:50,  1.56it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<04:34,  1.31it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<04:39,  1.29it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<03:33,  1.68it/s, Materializing param=model.layers.7.mlp.up_proj.weight]  
Loading weights:  19%|█▊        | 83/443 [00:54<03:33,  1.68it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<03:13,  1.86it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
Loading weights:  19%|█▊        | 83/443 [00:54<03:13,  1.86it/s, Materializing param=model.layers.7.mlp.up_proj.weight]  
Loading weights:  19%|█▊        | 83/443 [00:54<03:13,  1.86it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<04:45,  1.26it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<04:18,  1.39it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<04:55,  1.21it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<04:18,  1.39it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<04:55,  1.21it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<04:18,  1.39it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<04:55,  1.21it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<05:19,  1.13it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<04:17,  1.39it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]        
Loading weights:  19%|█▉        | 85/443 [00:56<04:54,  1.21it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]        
Loading weights:  19%|█▉        | 84/443 [00:56<04:45,  1.26it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<04:17,  1.39it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<05:19,  1.13it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<05:59,  1.00s/it, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<04:54,  1.21it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<05:19,  1.13it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<05:20,  1.12it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<06:02,  1.01s/it, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<04:16,  1.39it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<05:59,  1.00s/it, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<05:57,  1.00it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<04:45,  1.26it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<05:20,  1.12it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<04:16,  1.39it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<05:59,  1.00s/it, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<05:20,  1.12it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<05:57,  1.00it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<06:02,  1.01s/it, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<05:58,  1.00s/it, Materializing param=model.layers.7.self_attn.k_norm.weight]        
Loading weights:  19%|█▉        | 84/443 [00:56<05:57,  1.00it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 84/443 [00:56<06:02,  1.01s/it, Materializing param=model.layers.7.post_attention_layernorm.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<05:58,  1.00s/it, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<05:19,  1.12it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]        
Loading weights:  19%|█▉        | 85/443 [00:56<04:44,  1.26it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]        
Loading weights:  19%|█▉        | 85/443 [00:56<05:19,  1.12it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<05:18,  1.13it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]        
Loading weights:  19%|█▉        | 86/443 [00:56<05:57,  1.00s/it, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<05:56,  1.00it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]        
Loading weights:  19%|█▉        | 85/443 [00:56<05:18,  1.13it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<05:57,  1.00s/it, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<06:01,  1.01s/it, Materializing param=model.layers.7.self_attn.k_norm.weight]        
Loading weights:  19%|█▉        | 86/443 [00:56<05:18,  1.12it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<04:44,  1.26it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<05:56,  1.00it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<05:18,  1.12it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 85/443 [00:56<06:01,  1.01s/it, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<05:17,  1.13it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<05:17,  1.13it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<05:55,  1.00it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<05:55,  1.00it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<04:53,  1.21it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<06:00,  1.01s/it, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<04:53,  1.21it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<06:00,  1.01s/it, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:07,  1.90it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<02:57,  2.00it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<02:59,  1.99it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:00,  1.97it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:08,  1.89it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:06,  1.91it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<02:59,  1.99it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:07,  1.90it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<03:18,  1.80it/s, Materializing param=model.layers.7.self_attn.k_norm.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:06,  1.91it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:00,  1.97it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<02:57,  2.00it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:07,  1.90it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<02:59,  1.99it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:08,  1.89it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:06,  1.91it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:00,  1.97it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<02:57,  2.00it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:08,  1.89it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<03:18,  1.80it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  19%|█▉        | 86/443 [00:56<03:18,  1.80it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:18,  1.80it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<03:18,  1.80it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<02:43,  2.18it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<02:43,  2.18it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 87/443 [00:56<02:43,  2.18it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<02:50,  2.08it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<02:59,  1.97it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<02:59,  1.97it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<02:58,  1.99it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<03:06,  1.90it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<02:58,  1.99it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<03:06,  1.90it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<03:06,  1.91it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<02:50,  2.08it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<03:06,  1.91it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:40,  2.20it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:40,  2.20it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<02:42,  2.18it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<02:50,  2.08it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:40,  2.20it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<02:57,  2.00it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<02:42,  2.18it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<02:57,  2.00it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<03:07,  1.89it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:29,  2.37it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|█▉        | 88/443 [00:57<03:07,  1.89it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:29,  2.37it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:37,  2.25it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:29,  2.37it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:37,  2.25it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:43,  2.17it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:37,  2.25it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:43,  2.17it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:43,  2.17it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:42,  2.18it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:42,  2.18it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:42,  2.18it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:44,  2.15it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:44,  2.15it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:44,  2.15it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:32,  2.31it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:32,  2.31it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<03:05,  1.91it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:45,  2.14it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:32,  2.31it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:40,  2.20it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:45,  2.14it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<03:10,  1.85it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:40,  2.20it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:45,  2.14it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:40,  2.20it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<03:05,  1.91it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<03:10,  1.85it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<03:05,  1.91it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<03:10,  1.85it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<03:10,  1.85it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<03:04,  1.91it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<03:10,  1.85it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<03:04,  1.91it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:51,  2.06it/s, Materializing param=model.layers.7.self_attn.q_norm.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:51,  2.06it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 89/443 [00:57<02:51,  2.06it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:51,  2.06it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 90/443 [00:57<02:51,  2.06it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  21%|██        | 91/443 [00:57<02:50,  2.06it/s, Materializing param=model.layers.8.input_layernorm.weight] 
Loading weights:  21%|██        | 91/443 [00:57<02:50,  2.06it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  21%|██        | 92/443 [00:57<02:50,  2.06it/s, Materializing param=model.layers.8.mlp.down_proj.weight]  
Loading weights:  21%|██        | 92/443 [00:57<02:50,  2.06it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 91/443 [00:57<02:44,  2.15it/s, Materializing param=model.layers.8.input_layernorm.weight] 
Loading weights:  21%|██        | 91/443 [00:57<02:44,  2.15it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  21%|██        | 91/443 [00:57<02:32,  2.31it/s, Materializing param=model.layers.8.input_layernorm.weight] 
Loading weights:  21%|██        | 92/443 [00:57<02:43,  2.15it/s, Materializing param=model.layers.8.mlp.down_proj.weight]  
Loading weights:  21%|██        | 92/443 [00:57<02:43,  2.15it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 91/443 [00:57<02:32,  2.31it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  21%|██        | 92/443 [00:57<02:32,  2.31it/s, Materializing param=model.layers.8.mlp.down_proj.weight]  
Loading weights:  21%|██        | 92/443 [00:57<02:32,  2.31it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 91/443 [00:57<02:39,  2.20it/s, Materializing param=model.layers.8.input_layernorm.weight] 
Loading weights:  21%|██        | 91/443 [00:57<02:39,  2.20it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  21%|██        | 92/443 [00:57<02:39,  2.20it/s, Materializing param=model.layers.8.mlp.down_proj.weight]  
Loading weights:  21%|██        | 92/443 [00:57<02:39,  2.20it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 91/443 [00:57<03:09,  1.85it/s, Materializing param=model.layers.8.input_layernorm.weight] 
Loading weights:  21%|██        | 91/443 [00:57<03:09,  1.85it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  21%|██        | 92/443 [00:57<03:09,  1.85it/s, Materializing param=model.layers.8.mlp.down_proj.weight]  
Loading weights:  21%|██        | 91/443 [00:57<02:44,  2.14it/s, Materializing param=model.layers.8.input_layernorm.weight] 
Loading weights:  21%|██        | 92/443 [00:57<03:09,  1.85it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 91/443 [00:57<02:44,  2.14it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  21%|██        | 91/443 [00:57<03:03,  1.91it/s, Materializing param=model.layers.8.input_layernorm.weight] 
Loading weights:  21%|██        | 91/443 [00:57<03:03,  1.91it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  21%|██        | 92/443 [00:57<03:03,  1.91it/s, Materializing param=model.layers.8.mlp.down_proj.weight]  
Loading weights:  21%|██        | 92/443 [00:57<03:03,  1.91it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:07,  2.75it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:11,  2.66it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<01:58,  2.96it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:11,  2.66it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:10,  2.69it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:09,  2.69it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:07,  2.75it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:11,  2.66it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:11,  2.66it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:10,  2.69it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:09,  2.69it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:07,  2.75it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:10,  2.69it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:09,  2.69it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:11,  2.66it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<02:11,  2.66it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<01:58,  2.96it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<01:58,  2.96it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  20%|██        | 90/443 [00:58<02:40,  2.20it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  20%|██        | 90/443 [00:58<02:40,  2.20it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  21%|██        | 91/443 [00:58<03:01,  1.94it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
Loading weights:  21%|██        | 91/443 [00:58<03:01,  1.94it/s, Materializing param=model.layers.8.input_layernorm.weight] 
Loading weights:  21%|██        | 91/443 [00:58<03:01,  1.94it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  21%|██        | 92/443 [00:58<03:00,  1.94it/s, Materializing param=model.layers.8.mlp.down_proj.weight]  
Loading weights:  21%|██        | 92/443 [00:58<03:00,  1.94it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<03:00,  1.94it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [00:58<03:00,  1.94it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:29,  1.67it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:34,  1.63it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:13,  1.80it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:44,  1.55it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:26,  1.69it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:41,  1.58it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:34,  1.63it/s, Materializing param=model.layers.8.mlp.up_proj.weight]  
Loading weights:  21%|██        | 94/443 [01:00<03:13,  1.80it/s, Materializing param=model.layers.8.mlp.up_proj.weight]  
Loading weights:  21%|██        | 94/443 [01:00<03:29,  1.67it/s, Materializing param=model.layers.8.mlp.up_proj.weight]  
Loading weights:  21%|██        | 94/443 [01:00<03:44,  1.55it/s, Materializing param=model.layers.8.mlp.up_proj.weight]  
Loading weights:  21%|██        | 94/443 [01:00<03:22,  1.72it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:34,  1.63it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:26,  1.69it/s, Materializing param=model.layers.8.mlp.up_proj.weight]  
Loading weights:  21%|██        | 94/443 [01:00<03:13,  1.80it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:29,  1.67it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:41,  1.58it/s, Materializing param=model.layers.8.mlp.up_proj.weight]  
Loading weights:  21%|██        | 94/443 [01:00<03:44,  1.55it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:26,  1.69it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:41,  1.58it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:00<03:28,  1.67it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  21%|██▏       | 95/443 [01:00<03:28,  1.67it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:22,  1.72it/s, Materializing param=model.layers.8.mlp.up_proj.weight]  
Loading weights:  22%|██▏       | 96/443 [01:00<03:28,  1.67it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]        
Loading weights:  22%|██▏       | 96/443 [01:00<03:28,  1.67it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights:  21%|██        | 94/443 [01:00<03:22,  1.72it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  22%|██▏       | 97/443 [01:00<03:27,  1.67it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 97/443 [01:00<03:27,  1.67it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:00<03:26,  1.67it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:00<03:26,  1.67it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:02,  2.82it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:02,  2.82it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:02,  2.82it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:01,  2.82it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:01,  2.82it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:39,  1.58it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:39,  1.58it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:39,  1.58it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  22%|██▏       | 96/443 [01:01<03:39,  1.58it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]        
Loading weights:  22%|██▏       | 96/443 [01:01<03:39,  1.58it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<03:38,  1.58it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<03:38,  1.58it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:16,  1.77it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:16,  1.77it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:16,  1.77it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  22%|██▏       | 96/443 [01:01<03:16,  1.77it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]        
Loading weights:  22%|██▏       | 96/443 [01:01<03:16,  1.77it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:27,  1.68it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:27,  1.68it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:27,  1.68it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  22%|██▏       | 96/443 [01:01<03:26,  1.68it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]        
Loading weights:  22%|██▏       | 96/443 [01:01<03:26,  1.68it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<03:26,  1.68it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<03:26,  1.68it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:42,  1.56it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:42,  1.56it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:42,  1.56it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  22%|██▏       | 96/443 [01:01<03:42,  1.56it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]        
Loading weights:  22%|██▏       | 96/443 [01:01<03:42,  1.56it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<03:41,  1.56it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<03:41,  1.56it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  21%|██        | 92/443 [01:01<05:10,  1.13it/s, Materializing param=model.layers.8.input_layernorm.weight]
Loading weights:  21%|██        | 92/443 [01:01<05:10,  1.13it/s, Materializing param=model.layers.8.mlp.down_proj.weight]  
Loading weights:  21%|██        | 92/443 [01:01<05:10,  1.13it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
Loading weights:  21%|██        | 93/443 [01:01<05:09,  1.13it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 93/443 [01:01<05:09,  1.13it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
Loading weights:  21%|██        | 94/443 [01:01<05:08,  1.13it/s, Materializing param=model.layers.8.mlp.up_proj.weight]  
Loading weights:  21%|██        | 94/443 [01:01<05:08,  1.13it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<05:07,  1.13it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<05:07,  1.13it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  22%|██▏       | 96/443 [01:01<05:06,  1.13it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]        
Loading weights:  22%|██▏       | 96/443 [01:01<05:06,  1.13it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<05:05,  1.13it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<05:05,  1.13it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:25,  1.69it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:25,  1.69it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:25,  1.69it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  22%|██▏       | 96/443 [01:01<03:24,  1.69it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]        
Loading weights:  22%|██▏       | 96/443 [01:01<03:24,  1.69it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<03:24,  1.69it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<03:24,  1.69it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:35,  1.61it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:35,  1.61it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  21%|██▏       | 95/443 [01:01<03:35,  1.61it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
Loading weights:  22%|██▏       | 96/443 [01:01<03:35,  1.61it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]        
Loading weights:  22%|██▏       | 96/443 [01:01<03:35,  1.61it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<03:34,  1.61it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<03:34,  1.61it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<02:27,  2.34it/s, Materializing param=model.layers.8.self_attn.k_norm.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<02:27,  2.34it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 97/443 [01:01<02:27,  2.34it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:27,  2.34it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:27,  2.34it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.65it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:09,  2.67it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.64it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:09,  2.67it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:05,  2.74it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:09,  2.67it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.64it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.64it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.65it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.64it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.64it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:05,  2.74it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:10,  2.64it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 98/443 [01:01<02:05,  2.74it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:45,  3.25it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:45,  3.25it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:45,  3.25it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:26,  2.34it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:26,  2.34it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:26,  2.34it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:26,  2.34it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 102/443 [01:01<01:41,  3.35it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 102/443 [01:01<01:41,  3.35it/s, Materializing param=model.layers.9.input_layernorm.weight] 
Loading weights:  23%|██▎       | 102/443 [01:01<01:41,  3.35it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██▎       | 103/443 [01:01<01:41,  3.35it/s, Materializing param=model.layers.9.mlp.down_proj.weight]  
Loading weights:  23%|██▎       | 103/443 [01:01<01:41,  3.35it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:09,  2.65it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:09,  2.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:09,  2.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:09,  2.65it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:09,  2.65it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:11,  2.62it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:11,  2.62it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:11,  2.62it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:10,  2.62it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:10,  2.62it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<02:10,  2.62it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<02:10,  2.62it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 102/443 [01:01<02:10,  2.62it/s, Materializing param=model.layers.9.input_layernorm.weight] 
Loading weights:  23%|██▎       | 102/443 [01:01<02:10,  2.62it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██▎       | 103/443 [01:01<02:09,  2.62it/s, Materializing param=model.layers.9.mlp.down_proj.weight]  
Loading weights:  23%|██▎       | 103/443 [01:01<02:09,  2.62it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:35,  3.57it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:35,  3.57it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:35,  3.57it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 102/443 [01:01<01:35,  3.57it/s, Materializing param=model.layers.9.input_layernorm.weight] 
Loading weights:  23%|██▎       | 102/443 [01:01<01:35,  3.57it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:14,  2.56it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:14,  2.56it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:14,  2.56it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:14,  2.56it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:14,  2.56it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:11,  2.61it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:11,  2.61it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:11,  2.61it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:09,  2.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:09,  2.65it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<01:58,  2.89it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<01:58,  2.89it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<01:58,  2.89it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:16,  2.52it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:16,  2.52it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  22%|██▏       | 99/443 [01:01<02:16,  2.52it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:15,  2.52it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<02:15,  2.52it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:36,  3.54it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:36,  3.54it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:36,  3.54it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 102/443 [01:01<01:36,  3.54it/s, Materializing param=model.layers.9.input_layernorm.weight] 
Loading weights:  23%|██▎       | 102/443 [01:01<01:36,  3.54it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:37,  3.49it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:37,  3.49it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:37,  3.49it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:38,  3.49it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:38,  3.49it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:38,  3.49it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:49,  3.13it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:49,  3.13it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:49,  3.13it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<01:56,  2.96it/s, Materializing param=model.layers.8.self_attn.q_norm.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<01:56,  2.96it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 100/443 [01:01<01:56,  2.96it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:55,  2.96it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 101/443 [01:01<01:55,  2.96it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
Loading weights:  23%|██▎       | 102/443 [01:01<01:37,  3.49it/s, Materializing param=model.layers.9.input_layernorm.weight] 
Loading weights:  23%|██▎       | 102/443 [01:01<01:37,  3.49it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██▎       | 102/443 [01:01<01:37,  3.49it/s, Materializing param=model.layers.9.input_layernorm.weight] 
Loading weights:  23%|██▎       | 102/443 [01:01<01:48,  3.13it/s, Materializing param=model.layers.9.input_layernorm.weight] 
Loading weights:  23%|██▎       | 102/443 [01:01<01:48,  3.13it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██▎       | 102/443 [01:01<01:37,  3.49it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██▎       | 103/443 [01:01<01:37,  3.49it/s, Materializing param=model.layers.9.mlp.down_proj.weight]  
Loading weights:  23%|██▎       | 103/443 [01:01<01:37,  3.49it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██▎       | 102/443 [01:01<01:55,  2.96it/s, Materializing param=model.layers.9.input_layernorm.weight] 
Loading weights:  23%|██▎       | 102/443 [01:01<01:55,  2.96it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██▎       | 103/443 [01:01<01:55,  2.96it/s, Materializing param=model.layers.9.mlp.down_proj.weight]  
Loading weights:  23%|██▎       | 103/443 [01:01<01:55,  2.96it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:46,  2.04it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:05,  2.70it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:06,  2.67it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:46,  2.04it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:12,  2.56it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:46,  2.04it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:06,  2.67it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 103/443 [01:03<02:33,  2.21it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:06,  2.67it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:12,  2.56it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 103/443 [01:03<02:34,  2.20it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:12,  2.56it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 103/443 [01:03<02:33,  2.21it/s, Materializing param=model.layers.9.mlp.down_proj.weight]  
Loading weights:  23%|██▎       | 104/443 [01:03<02:05,  2.70it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 103/443 [01:03<02:34,  2.20it/s, Materializing param=model.layers.9.mlp.down_proj.weight]  
Loading weights:  23%|██▎       | 103/443 [01:03<02:33,  2.21it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██▎       | 103/443 [01:03<02:34,  2.20it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:34,  2.20it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:33,  2.21it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:05,  2.70it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 103/443 [01:03<02:27,  2.30it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:34,  2.20it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:33,  2.21it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 103/443 [01:03<02:36,  2.18it/s, Materializing param=model.layers.9.input_layernorm.weight]
Loading weights:  23%|██▎       | 103/443 [01:03<02:27,  2.30it/s, Materializing param=model.layers.9.mlp.down_proj.weight]  
Loading weights:  23%|██▎       | 103/443 [01:03<02:27,  2.30it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██▎       | 103/443 [01:03<02:36,  2.18it/s, Materializing param=model.layers.9.mlp.down_proj.weight]  
Loading weights:  24%|██▎       | 105/443 [01:03<02:05,  2.70it/s, Materializing param=model.layers.9.mlp.up_proj.weight]  
Loading weights:  23%|██▎       | 103/443 [01:03<02:36,  2.18it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:35,  2.18it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:35,  2.18it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:27,  2.30it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:03<02:05,  2.70it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  23%|██▎       | 104/443 [01:03<02:27,  2.30it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<01:57,  2.86it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<01:57,  2.86it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<01:57,  2.86it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 107/443 [01:04<01:57,  2.86it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]        
Loading weights:  24%|██▍       | 107/443 [01:04<01:57,  2.86it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<01:57,  2.86it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<01:57,  2.86it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:18,  2.45it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:18,  2.45it/s, Materializing param=model.layers.9.mlp.up_proj.weight]  
Loading weights:  24%|██▎       | 105/443 [01:04<02:18,  2.45it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:24,  2.34it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:15,  2.50it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:19,  2.43it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:24,  2.34it/s, Materializing param=model.layers.9.mlp.up_proj.weight]  
Loading weights:  24%|██▎       | 105/443 [01:04<02:15,  2.50it/s, Materializing param=model.layers.9.mlp.up_proj.weight]  
Loading weights:  24%|██▎       | 105/443 [01:04<02:24,  2.34it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:15,  2.50it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:19,  2.43it/s, Materializing param=model.layers.9.mlp.up_proj.weight]  
Loading weights:  24%|██▎       | 105/443 [01:04<02:19,  2.43it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:17,  2.46it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:17,  2.46it/s, Materializing param=model.layers.9.mlp.up_proj.weight]  
Loading weights:  24%|██▎       | 105/443 [01:04<02:17,  2.46it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:52,  1.96it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:52,  1.96it/s, Materializing param=model.layers.9.mlp.up_proj.weight]  
Loading weights:  24%|██▎       | 105/443 [01:04<02:52,  1.96it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:17,  2.46it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
Loading weights:  24%|██▎       | 105/443 [01:04<02:17,  2.46it/s, Materializing param=model.layers.9.mlp.up_proj.weight]  
Loading weights:  24%|██▎       | 105/443 [01:04<02:17,  2.46it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<01:18,  4.26it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<01:18,  4.26it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<01:18,  4.26it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:50,  1.98it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:50,  1.98it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:50,  1.98it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 107/443 [01:04<02:49,  1.98it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]        
Loading weights:  24%|██▍       | 107/443 [01:04<02:49,  1.98it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:49,  1.98it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:49,  1.98it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:20,  2.40it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:20,  2.40it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:20,  2.40it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 107/443 [01:04<02:19,  2.40it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]        
Loading weights:  24%|██▍       | 107/443 [01:04<02:19,  2.40it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:19,  2.40it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:19,  2.40it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:48,  1.98it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:48,  1.98it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:23,  2.35it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:23,  2.35it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:23,  2.35it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 107/443 [01:04<02:22,  2.35it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]        
Loading weights:  24%|██▍       | 107/443 [01:04<02:22,  2.35it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:22,  2.35it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:22,  2.35it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:19,  2.40it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:19,  2.40it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:22,  2.35it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:22,  2.35it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:31,  2.23it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:32,  2.21it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:31,  2.23it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:31,  2.23it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:32,  2.21it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:32,  2.21it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 107/443 [01:04<02:30,  2.23it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]        
Loading weights:  24%|██▍       | 107/443 [01:04<02:30,  2.23it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights:  24%|██▍       | 107/443 [01:04<02:32,  2.21it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]        
Loading weights:  24%|██▍       | 107/443 [01:04<02:32,  2.21it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:30,  2.23it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:30,  2.23it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:31,  2.21it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:31,  2.21it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:32,  2.22it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:32,  2.22it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:32,  2.22it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 107/443 [01:04<02:31,  2.22it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]        
Loading weights:  24%|██▍       | 107/443 [01:04<02:31,  2.22it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:31,  2.22it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:31,  2.22it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:38,  2.12it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:29,  2.23it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:38,  2.12it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:29,  2.23it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  24%|██▍       | 106/443 [01:04<02:38,  2.12it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
Loading weights:  24%|██▍       | 107/443 [01:04<02:38,  2.12it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]        
Loading weights:  24%|██▍       | 107/443 [01:04<02:38,  2.12it/s, Materializing param=model.layers.9.self_attn.k_norm.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:37,  2.12it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  24%|██▍       | 108/443 [01:04<02:37,  2.12it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:31,  2.21it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:31,  2.21it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:30,  2.22it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:30,  2.22it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:37,  2.12it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 109/443 [01:04<02:37,  2.12it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:28,  3.76it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:28,  3.76it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:28,  3.76it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:19,  4.18it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:19,  4.18it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:18,  4.26it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:28,  3.76it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:19,  4.18it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:28,  3.76it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:18,  4.26it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:23,  4.00it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:23,  4.00it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:23,  4.00it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:17,  4.28it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:17,  4.28it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:17,  4.28it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:17,  4.28it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:17,  4.28it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:20,  4.15it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:20,  4.15it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:20,  4.15it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:20,  4.15it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:20,  4.15it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:22,  4.05it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:22,  4.05it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:22,  4.05it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:21,  4.05it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:21,  4.05it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:20,  4.12it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:20,  4.12it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▍       | 110/443 [01:04<01:20,  4.12it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:20,  4.12it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 111/443 [01:04<01:20,  4.12it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:06,  4.96it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:16,  4.30it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:06,  5.00it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:24,  3.91it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:06,  4.98it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 111/443 [01:05<01:21,  4.07it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:06,  4.96it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:16,  4.30it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:06,  5.00it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:06,  4.98it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:06,  4.96it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:16,  4.30it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:06,  5.00it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:06,  4.98it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 111/443 [01:05<01:21,  4.07it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 111/443 [01:05<01:21,  4.07it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:24,  3.91it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:21,  4.07it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:24,  3.91it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:21,  4.07it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:08,  4.84it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:08,  4.84it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:08,  4.84it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:05<01:32,  3.58it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
Loading weights:  25%|██▍       | 110/443 [01:05<01:32,  3.58it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▍       | 110/443 [01:05<01:32,  3.58it/s, Materializing param=model.layers.9.self_attn.q_norm.weight]
Loading weights:  25%|██▌       | 111/443 [01:05<01:32,  3.58it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 111/443 [01:05<01:32,  3.58it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:32,  3.58it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  25%|██▌       | 112/443 [01:05<01:32,  3.58it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:06,  4.98it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:06,  4.98it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:05,  5.00it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:10,  4.70it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:05,  5.00it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:15,  4.36it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:02,  5.28it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:10,  4.70it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:10,  4.70it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:15,  4.36it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:15,  4.36it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<00:52,  6.24it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:02,  5.28it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:02,  5.28it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<00:52,  6.24it/s, Materializing param=model.layers.10.mlp.down_proj.weight]  
Loading weights:  26%|██▌       | 114/443 [01:05<00:52,  6.24it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<01:15,  4.36it/s, Materializing param=model.layers.10.mlp.down_proj.weight]  
Loading weights:  26%|██▌       | 114/443 [01:05<01:02,  5.28it/s, Materializing param=model.layers.10.mlp.down_proj.weight]  
Loading weights:  26%|██▌       | 113/443 [01:05<01:06,  4.96it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<01:15,  4.36it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:06,  4.96it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<01:02,  5.28it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:08,  4.84it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:08,  4.84it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<00:53,  6.19it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<00:53,  6.19it/s, Materializing param=model.layers.10.mlp.down_proj.weight]  
Loading weights:  26%|██▌       | 114/443 [01:05<00:53,  6.19it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<00:54,  6.03it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<00:54,  6.03it/s, Materializing param=model.layers.10.mlp.down_proj.weight]  
Loading weights:  26%|██▌       | 114/443 [01:05<00:54,  6.03it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<01:43,  3.17it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<01:27,  3.77it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<01:43,  3.17it/s, Materializing param=model.layers.10.mlp.down_proj.weight]  
Loading weights:  26%|██▌       | 114/443 [01:05<01:43,  3.17it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<01:27,  3.77it/s, Materializing param=model.layers.10.mlp.down_proj.weight]  
Loading weights:  26%|██▌       | 115/443 [01:05<01:43,  3.17it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<01:43,  3.17it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<01:27,  3.77it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<00:54,  6.03it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<01:20,  4.06it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<00:52,  6.24it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<00:53,  6.19it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<00:54,  6.03it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<01:20,  4.06it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<00:52,  6.24it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<01:32,  3.56it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<01:20,  4.06it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<00:53,  6.19it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<01:26,  3.77it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<01:26,  3.77it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<01:32,  3.56it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<01:32,  3.56it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:32,  3.58it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:32,  3.58it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 113/443 [01:05<01:32,  3.58it/s, Materializing param=model.layers.10.input_layernorm.weight]
Loading weights:  26%|██▌       | 114/443 [01:05<01:31,  3.58it/s, Materializing param=model.layers.10.mlp.down_proj.weight]  
Loading weights:  26%|██▌       | 114/443 [01:05<01:31,  3.58it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<01:31,  3.58it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 115/443 [01:05<01:31,  3.58it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:42,  2.01it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:29,  2.18it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:19,  2.35it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:00,  2.72it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:30,  2.17it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:41,  2.02it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:42,  2.01it/s, Materializing param=model.layers.10.mlp.up_proj.weight]  
Loading weights:  26%|██▌       | 116/443 [01:07<02:29,  2.18it/s, Materializing param=model.layers.10.mlp.up_proj.weight]  
Loading weights:  26%|██▌       | 116/443 [01:07<02:00,  2.72it/s, Materializing param=model.layers.10.mlp.up_proj.weight]  
Loading weights:  26%|██▌       | 116/443 [01:07<02:30,  2.17it/s, Materializing param=model.layers.10.mlp.up_proj.weight]  
Loading weights:  26%|██▌       | 116/443 [01:07<02:19,  2.35it/s, Materializing param=model.layers.10.mlp.up_proj.weight]  
Loading weights:  26%|██▌       | 116/443 [01:07<02:42,  2.01it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:41,  2.02it/s, Materializing param=model.layers.10.mlp.up_proj.weight]  
Loading weights:  26%|██▌       | 116/443 [01:07<02:29,  2.18it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:00,  2.72it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:30,  2.17it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:41,  2.02it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:19,  2.35it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<03:00,  1.81it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<03:00,  1.81it/s, Materializing param=model.layers.10.mlp.up_proj.weight]  
Loading weights:  26%|██▌       | 116/443 [01:07<03:00,  1.81it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:29,  2.19it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
Loading weights:  26%|██▌       | 116/443 [01:07<02:29,  2.19it/s, Materializing param=model.layers.10.mlp.up_proj.weight]  
Loading weights:  26%|██▌       | 116/443 [01:07<02:29,  2.19it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:29,  2.19it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:29,  2.19it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  27%|██▋       | 118/443 [01:07<02:28,  2.19it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]        
Loading weights:  27%|██▋       | 118/443 [01:07<02:28,  2.19it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:11,  2.47it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:28,  2.20it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<01:56,  2.79it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:20,  2.32it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:11,  2.47it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:28,  2.20it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:11,  2.47it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<01:56,  2.79it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:28,  2.20it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:42,  2.01it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:20,  2.32it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:28,  2.19it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:20,  2.32it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<01:56,  2.79it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:28,  2.19it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:28,  2.19it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  27%|██▋       | 118/443 [01:07<02:28,  2.19it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]        
Loading weights:  27%|██▋       | 118/443 [01:07<02:11,  2.47it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]        
Loading weights:  27%|██▋       | 118/443 [01:07<02:28,  2.19it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights:  27%|██▋       | 118/443 [01:07<02:11,  2.47it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:42,  2.01it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:42,  2.01it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  27%|██▋       | 118/443 [01:07<02:27,  2.20it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]        
Loading weights:  27%|██▋       | 118/443 [01:07<01:56,  2.79it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]        
Loading weights:  27%|██▋       | 118/443 [01:07<02:20,  2.32it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]        
Loading weights:  27%|██▋       | 118/443 [01:07<02:27,  2.20it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights:  27%|██▋       | 118/443 [01:07<01:56,  2.79it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights:  27%|██▋       | 118/443 [01:07<02:20,  2.32it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:21,  2.31it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
Loading weights:  27%|██▋       | 118/443 [01:07<02:41,  2.01it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]        
Loading weights:  26%|██▋       | 117/443 [01:07<02:21,  2.31it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  26%|██▋       | 117/443 [01:07<02:21,  2.31it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<01:56,  2.79it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<01:56,  2.79it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<02:27,  2.20it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<02:19,  2.32it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 118/443 [01:07<02:20,  2.31it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]        
Loading weights:  27%|██▋       | 119/443 [01:07<02:27,  2.20it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 118/443 [01:07<02:41,  2.01it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights:  27%|██▋       | 118/443 [01:07<02:20,  2.31it/s, Materializing param=model.layers.10.self_attn.k_norm.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<02:19,  2.32it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<02:20,  2.31it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<02:20,  2.31it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<02:41,  2.01it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<02:41,  2.01it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<02:10,  2.47it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<02:28,  2.19it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<02:10,  2.47it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 119/443 [01:07<02:28,  2.19it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<02:19,  2.32it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<02:19,  2.32it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<02:26,  2.20it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<02:26,  2.20it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<01:55,  2.79it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<01:55,  2.79it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<02:10,  2.47it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<02:10,  2.47it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<02:27,  2.19it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<02:27,  2.19it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<02:19,  2.31it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:07<02:19,  2.31it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:18,  4.09it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:20,  4.01it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:18,  4.08it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:29,  3.60it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:15,  4.25it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:20,  4.01it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:18,  4.08it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:15,  4.25it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:29,  3.60it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:18,  4.08it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:20,  4.01it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:18,  4.09it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:15,  4.25it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:29,  3.60it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:18,  4.09it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:18,  4.08it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:15,  4.25it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:18,  4.08it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:29,  3.60it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:15,  4.25it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:29,  3.60it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:18,  4.09it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:18,  4.09it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:20,  4.01it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:20,  4.01it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:08<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:08<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 120/443 [01:08<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:36,  3.34it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:10,  4.57it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:08,  4.69it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:09,  4.58it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:08,  4.67it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:11,  4.47it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:08,  4.69it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:10,  4.57it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:09,  4.58it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:08,  4.69it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:10,  4.57it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:09,  4.58it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:11,  4.47it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:11,  4.47it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:08,  4.67it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:08,  4.67it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:17,  4.11it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:17,  4.11it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:17,  4.11it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 124/443 [01:08<01:17,  4.11it/s, Materializing param=model.layers.11.input_layernorm.weight] 
Loading weights:  28%|██▊       | 124/443 [01:08<01:17,  4.11it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|██▊       | 124/443 [01:08<01:09,  4.58it/s, Materializing param=model.layers.11.input_layernorm.weight] 
Loading weights:  28%|██▊       | 124/443 [01:08<01:09,  4.58it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|██▊       | 124/443 [01:08<01:08,  4.67it/s, Materializing param=model.layers.11.input_layernorm.weight] 
Loading weights:  28%|██▊       | 124/443 [01:08<01:08,  4.67it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|██▊       | 125/443 [01:08<01:08,  4.67it/s, Materializing param=model.layers.11.mlp.down_proj.weight]  
Loading weights:  28%|██▊       | 125/443 [01:08<01:08,  4.67it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 124/443 [01:08<01:08,  4.69it/s, Materializing param=model.layers.11.input_layernorm.weight] 
Loading weights:  28%|██▊       | 124/443 [01:08<01:08,  4.69it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|██▊       | 125/443 [01:08<01:07,  4.69it/s, Materializing param=model.layers.11.mlp.down_proj.weight]  
Loading weights:  28%|██▊       | 125/443 [01:08<01:07,  4.69it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 124/443 [01:08<01:11,  4.47it/s, Materializing param=model.layers.11.input_layernorm.weight] 
Loading weights:  28%|██▊       | 124/443 [01:08<01:11,  4.47it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|██▊       | 124/443 [01:08<01:09,  4.57it/s, Materializing param=model.layers.11.input_layernorm.weight] 
Loading weights:  28%|██▊       | 124/443 [01:08<01:09,  4.57it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:28,  3.65it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:36,  3.33it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:28,  3.65it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:28,  3.65it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:36,  3.33it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  27%|██▋       | 121/443 [01:08<01:36,  3.33it/s, Materializing param=model.layers.10.self_attn.q_norm.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:27,  3.65it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:27,  3.65it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:27,  3.65it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:27,  3.65it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:36,  3.33it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 122/443 [01:08<01:36,  3.33it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:36,  3.33it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 123/443 [01:08<01:36,  3.33it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
Loading weights:  28%|██▊       | 124/443 [01:08<01:35,  3.33it/s, Materializing param=model.layers.11.input_layernorm.weight] 
Loading weights:  28%|██▊       | 124/443 [01:08<01:35,  3.33it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|██▊       | 124/443 [01:08<01:27,  3.65it/s, Materializing param=model.layers.11.input_layernorm.weight] 
Loading weights:  28%|██▊       | 124/443 [01:08<01:27,  3.65it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|██▊       | 125/443 [01:08<01:35,  3.33it/s, Materializing param=model.layers.11.mlp.down_proj.weight]  
Loading weights:  28%|██▊       | 125/443 [01:08<01:35,  3.33it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 125/443 [01:08<01:27,  3.65it/s, Materializing param=model.layers.11.mlp.down_proj.weight]  
Loading weights:  28%|██▊       | 125/443 [01:08<01:27,  3.65it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 125/443 [01:10<02:20,  2.27it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<01:42,  3.09it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:02,  2.58it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:01,  2.62it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 125/443 [01:10<02:22,  2.24it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<01:48,  2.93it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 125/443 [01:10<02:20,  2.27it/s, Materializing param=model.layers.11.mlp.down_proj.weight]  
Loading weights:  28%|██▊       | 126/443 [01:10<01:42,  3.09it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:01,  2.62it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 125/443 [01:10<02:27,  2.16it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  28%|██▊       | 125/443 [01:10<02:20,  2.27it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<01:48,  2.93it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:01,  2.62it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<01:42,  3.09it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<01:48,  2.93it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 125/443 [01:10<02:27,  2.16it/s, Materializing param=model.layers.11.mlp.down_proj.weight]  
Loading weights:  28%|██▊       | 125/443 [01:10<02:22,  2.24it/s, Materializing param=model.layers.11.mlp.down_proj.weight]  
Loading weights:  28%|██▊       | 125/443 [01:10<02:27,  2.16it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 125/443 [01:10<02:22,  2.24it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:02,  2.58it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:02,  2.58it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:26,  2.16it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:21,  2.24it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:26,  2.16it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:19,  2.27it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:21,  2.24it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:10<02:19,  2.27it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<02:34,  2.05it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<03:09,  1.67it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<03:05,  1.70it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<03:02,  1.73it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<03:05,  1.70it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<02:59,  1.76it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<02:34,  2.05it/s, Materializing param=model.layers.11.mlp.up_proj.weight]  
Loading weights:  29%|██▊       | 127/443 [01:12<03:05,  1.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight]  
Loading weights:  29%|██▊       | 127/443 [01:12<02:42,  1.95it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<02:34,  2.05it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<03:02,  1.73it/s, Materializing param=model.layers.11.mlp.up_proj.weight]  
Loading weights:  29%|██▊       | 127/443 [01:12<02:59,  1.76it/s, Materializing param=model.layers.11.mlp.up_proj.weight]  
Loading weights:  29%|██▊       | 127/443 [01:12<03:05,  1.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<03:05,  1.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight]  
Loading weights:  28%|██▊       | 125/443 [01:12<03:43,  1.42it/s, Materializing param=model.layers.11.input_layernorm.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<02:42,  1.95it/s, Materializing param=model.layers.11.mlp.up_proj.weight]  
Loading weights:  29%|██▊       | 127/443 [01:12<02:59,  1.76it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<03:02,  1.73it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<02:42,  1.95it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<03:05,  1.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  28%|██▊       | 125/443 [01:12<03:43,  1.42it/s, Materializing param=model.layers.11.mlp.down_proj.weight]  
Loading weights:  29%|██▊       | 127/443 [01:12<03:09,  1.67it/s, Materializing param=model.layers.11.mlp.up_proj.weight]  
Loading weights:  28%|██▊       | 125/443 [01:12<03:43,  1.42it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<03:09,  1.67it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:12<03:42,  1.42it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  28%|██▊       | 126/443 [01:12<03:42,  1.42it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
Loading weights:  29%|██▊       | 127/443 [01:12<03:41,  1.42it/s, Materializing param=model.layers.11.mlp.up_proj.weight]  
Loading weights:  29%|██▊       | 127/443 [01:12<03:41,  1.42it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:08,  1.27it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<03:41,  1.42it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:08,  1.27it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:08,  1.27it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<03:41,  1.42it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<03:41,  1.42it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:12,  1.25it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:16,  1.23it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:12,  1.25it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 129/443 [01:14<03:40,  1.42it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]        
Loading weights:  29%|██▉       | 128/443 [01:14<03:30,  1.50it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:12,  1.25it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:11,  1.25it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▉       | 129/443 [01:14<03:40,  1.42it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:04,  1.29it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<03:30,  1.50it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:11,  1.25it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<03:31,  1.49it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
Loading weights:  29%|██▉       | 129/443 [01:14<04:11,  1.25it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]        
Loading weights:  29%|██▉       | 128/443 [01:14<03:30,  1.50it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:11,  1.25it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:04,  1.29it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<03:31,  1.49it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<03:39,  1.42it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:16,  1.23it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 129/443 [01:14<04:11,  1.25it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<03:31,  1.49it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<03:39,  1.42it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:04,  1.29it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 129/443 [01:14<03:29,  1.50it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]        
Loading weights:  29%|██▉       | 129/443 [01:14<04:10,  1.25it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]        
Loading weights:  29%|██▉       | 129/443 [01:14<03:29,  1.50it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights:  29%|██▉       | 129/443 [01:14<04:10,  1.25it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights:  29%|██▉       | 129/443 [01:14<03:30,  1.49it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]        
Loading weights:  29%|██▉       | 129/443 [01:14<04:04,  1.29it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]        
Loading weights:  29%|██▉       | 129/443 [01:14<03:30,  1.49it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights:  29%|██▉       | 129/443 [01:14<04:04,  1.29it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights:  29%|██▉       | 128/443 [01:14<04:16,  1.23it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<04:09,  1.25it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<04:10,  1.25it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<04:09,  1.25it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<03:29,  1.49it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<04:03,  1.29it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<04:10,  1.25it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<03:29,  1.49it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<04:03,  1.29it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 129/443 [01:14<04:15,  1.23it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]        
Loading weights:  29%|██▉       | 129/443 [01:14<04:15,  1.23it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<04:14,  1.23it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<04:14,  1.23it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  29%|██▉       | 129/443 [01:14<03:26,  1.52it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
Loading weights:  29%|██▉       | 129/443 [01:14<03:26,  1.52it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]        
Loading weights:  29%|██▉       | 129/443 [01:14<03:26,  1.52it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:20,  2.22it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:30,  2.08it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:20,  2.22it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:26,  2.12it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:20,  2.22it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:26,  2.12it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:26,  2.12it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:29,  2.09it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:30,  2.08it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<02:38,  1.97it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:29,  2.09it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<02:38,  1.97it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:29,  2.09it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:30,  2.08it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<02:38,  1.97it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:37,  1.97it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:37,  1.97it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:29,  2.09it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:16,  2.29it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:29,  2.09it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:16,  2.29it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:29,  2.09it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<02:16,  2.29it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:33,  2.03it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:26,  2.12it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:25,  2.14it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:33,  2.03it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:25,  2.14it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:33,  2.03it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:26,  2.12it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:09,  2.39it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:32,  2.03it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:25,  2.14it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:09,  2.39it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:28,  2.09it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:22,  2.19it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:18,  2.25it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:09,  2.39it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:34,  2.02it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:32,  2.03it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:28,  2.09it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:22,  2.19it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<03:20,  1.56it/s, Materializing param=model.layers.11.self_attn.k_norm.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:18,  2.25it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:22,  2.19it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<03:20,  1.56it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:18,  2.25it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:24,  2.14it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:11,  2.36it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  29%|██▉       | 130/443 [01:14<03:20,  1.56it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:24,  2.14it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:21,  2.19it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:11,  2.36it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:21,  2.19it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:11,  2.36it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:18,  2.25it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:18,  2.25it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:34,  2.02it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<03:19,  1.56it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<02:34,  2.02it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|██▉       | 131/443 [01:14<03:19,  1.56it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<03:18,  1.56it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|██▉       | 132/443 [01:14<03:18,  1.56it/s, Materializing param=model.layers.11.self_attn.q_norm.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:33,  2.02it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<03:18,  1.56it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<03:18,  1.56it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 133/443 [01:14<02:33,  2.02it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:18,  2.24it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:19,  2.22it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:13,  2.32it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:06,  2.45it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:12,  2.33it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<01:53,  2.73it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:08,  2.41it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:06,  2.45it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:18,  2.24it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:12,  2.33it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:19,  2.22it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<01:53,  2.73it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:08,  2.41it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:18,  2.24it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:12,  2.33it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:19,  2.22it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<01:53,  2.73it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:08,  2.41it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:06,  2.45it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:13,  2.32it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:13,  2.32it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:04,  2.48it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:04,  2.48it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 134/443 [01:15<02:04,  2.48it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 135/443 [01:15<02:04,  2.48it/s, Materializing param=model.layers.12.input_layernorm.weight] 
Loading weights:  30%|███       | 135/443 [01:15<02:04,  2.48it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  31%|███       | 136/443 [01:15<02:03,  2.48it/s, Materializing param=model.layers.12.mlp.down_proj.weight]  
Loading weights:  31%|███       | 136/443 [01:15<02:03,  2.48it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  30%|███       | 135/443 [01:15<02:07,  2.41it/s, Materializing param=model.layers.12.input_layernorm.weight] 
Loading weights:  30%|███       | 135/443 [01:15<02:07,  2.41it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  30%|███       | 135/443 [01:15<02:17,  2.24it/s, Materializing param=model.layers.12.input_layernorm.weight] 
Loading weights:  30%|███       | 135/443 [01:15<02:17,  2.24it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  31%|███       | 136/443 [01:15<02:07,  2.41it/s, Materializing param=model.layers.12.mlp.down_proj.weight]  
Loading weights:  31%|███       | 136/443 [01:15<02:07,  2.41it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|███       | 136/443 [01:15<02:17,  2.24it/s, Materializing param=model.layers.12.mlp.down_proj.weight]  
Loading weights:  31%|███       | 136/443 [01:15<02:17,  2.24it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  30%|███       | 135/443 [01:15<02:12,  2.33it/s, Materializing param=model.layers.12.input_layernorm.weight] 
Loading weights:  30%|███       | 135/443 [01:15<02:12,  2.33it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  30%|███       | 135/443 [01:15<02:12,  2.32it/s, Materializing param=model.layers.12.input_layernorm.weight] 
Loading weights:  30%|███       | 135/443 [01:15<02:18,  2.22it/s, Materializing param=model.layers.12.input_layernorm.weight] 
Loading weights:  30%|███       | 135/443 [01:15<02:18,  2.22it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  31%|███       | 136/443 [01:15<02:18,  2.22it/s, Materializing param=model.layers.12.mlp.down_proj.weight]  
Loading weights:  31%|███       | 136/443 [01:15<02:18,  2.22it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  30%|███       | 135/443 [01:15<02:12,  2.32it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  31%|███       | 136/443 [01:15<02:12,  2.32it/s, Materializing param=model.layers.12.mlp.down_proj.weight]  
Loading weights:  30%|███       | 135/443 [01:15<02:05,  2.45it/s, Materializing param=model.layers.12.input_layernorm.weight] 
Loading weights:  30%|███       | 135/443 [01:15<02:05,  2.45it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  31%|███       | 136/443 [01:15<02:12,  2.32it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|███       | 136/443 [01:15<02:05,  2.45it/s, Materializing param=model.layers.12.mlp.down_proj.weight]  
Loading weights:  31%|███       | 136/443 [01:15<02:05,  2.45it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|███       | 136/443 [01:17<03:09,  1.62it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:43,  1.87it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:47,  1.82it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:35,  1.97it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:46,  1.83it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:37,  1.94it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:47,  1.82it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:39,  1.92it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:35,  1.97it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 136/443 [01:17<03:09,  1.62it/s, Materializing param=model.layers.12.mlp.down_proj.weight]  
Loading weights:  31%|███       | 137/443 [01:17<02:46,  1.83it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:37,  1.94it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:35,  1.97it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:47,  1.82it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 136/443 [01:17<03:09,  1.62it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:46,  1.83it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:39,  1.92it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:37,  1.94it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:39,  1.92it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:43,  1.87it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<03:08,  1.62it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<03:08,  1.62it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<02:43,  1.87it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  30%|███       | 135/443 [01:17<03:18,  1.55it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
Loading weights:  30%|███       | 135/443 [01:17<03:18,  1.55it/s, Materializing param=model.layers.12.input_layernorm.weight] 
Loading weights:  30%|███       | 135/443 [01:17<03:18,  1.55it/s, Materializing param=model.layers.12.input_layernorm.weight]
Loading weights:  31%|███       | 136/443 [01:17<03:18,  1.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight]  
Loading weights:  31%|███       | 136/443 [01:17<03:18,  1.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<03:17,  1.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 137/443 [01:17<03:17,  1.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<03:26,  1.48it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<03:58,  1.28it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<03:51,  1.32it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<03:26,  1.48it/s, Materializing param=model.layers.12.mlp.up_proj.weight]  
Loading weights:  31%|███       | 138/443 [01:19<03:56,  1.29it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<04:09,  1.22it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<03:52,  1.31it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<03:58,  1.28it/s, Materializing param=model.layers.12.mlp.up_proj.weight]  
Loading weights:  31%|███       | 138/443 [01:19<03:26,  1.48it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<03:51,  1.32it/s, Materializing param=model.layers.12.mlp.up_proj.weight]  
Loading weights:  31%|███       | 138/443 [01:19<04:04,  1.25it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<04:08,  1.23it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<03:58,  1.28it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<03:51,  1.32it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<03:56,  1.29it/s, Materializing param=model.layers.12.mlp.up_proj.weight]  
Loading weights:  31%|███       | 138/443 [01:19<03:52,  1.31it/s, Materializing param=model.layers.12.mlp.up_proj.weight]  
Loading weights:  31%|███       | 138/443 [01:19<03:56,  1.29it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<03:52,  1.31it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<04:09,  1.22it/s, Materializing param=model.layers.12.mlp.up_proj.weight]  
Loading weights:  31%|███       | 138/443 [01:19<04:08,  1.23it/s, Materializing param=model.layers.12.mlp.up_proj.weight]  
Loading weights:  31%|███       | 138/443 [01:19<04:09,  1.22it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<04:08,  1.23it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███       | 138/443 [01:19<04:04,  1.25it/s, Materializing param=model.layers.12.mlp.up_proj.weight]  
Loading weights:  31%|███▏      | 139/443 [01:19<03:55,  1.29it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  31%|███▏      | 139/443 [01:19<03:55,  1.29it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  31%|███       | 138/443 [01:19<04:04,  1.25it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  32%|███▏      | 140/443 [01:19<03:54,  1.29it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]        
Loading weights:  32%|███▏      | 140/443 [01:19<03:54,  1.29it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights:  32%|███▏      | 141/443 [01:19<03:53,  1.29it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 141/443 [01:19<03:53,  1.29it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:19<03:53,  1.29it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:19<03:53,  1.29it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:19<02:00,  2.50it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:19<02:00,  2.50it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  32%|███▏      | 143/443 [01:19<02:00,  2.50it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  33%|███▎      | 144/443 [01:19<01:59,  2.50it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 144/443 [01:19<01:59,  2.50it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:52,  1.31it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:52,  1.31it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:52,  1.31it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  32%|███▏      | 140/443 [01:20<03:51,  1.31it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]        
Loading weights:  32%|███▏      | 140/443 [01:20<03:51,  1.31it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<03:50,  1.31it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<03:50,  1.31it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:41,  2.93it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:41,  2.93it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:41,  2.93it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<03:50,  1.31it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<03:50,  1.31it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:45,  1.35it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:45,  1.35it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:45,  1.35it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  32%|███▏      | 140/443 [01:20<03:45,  1.35it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]        
Loading weights:  32%|███▏      | 140/443 [01:20<03:45,  1.35it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<03:44,  1.35it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<03:44,  1.35it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  33%|███▎      | 146/443 [01:20<01:41,  2.93it/s, Materializing param=model.layers.13.input_layernorm.weight] 
Loading weights:  33%|███▎      | 146/443 [01:20<01:41,  2.93it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:48,  1.33it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:48,  1.33it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:48,  1.33it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  32%|███▏      | 140/443 [01:20<03:47,  1.33it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]        
Loading weights:  32%|███▏      | 140/443 [01:20<03:47,  1.33it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<03:46,  1.33it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<03:46,  1.33it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<03:43,  1.35it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<03:43,  1.35it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<03:45,  1.33it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<03:45,  1.33it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:31,  1.44it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:31,  1.44it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:31,  1.44it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<04:06,  1.23it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  32%|███▏      | 140/443 [01:20<03:31,  1.44it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]        
Loading weights:  32%|███▏      | 140/443 [01:20<03:31,  1.44it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<04:06,  1.23it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<04:06,  1.23it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<03:30,  1.44it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<03:30,  1.44it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 140/443 [01:20<04:06,  1.23it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]        
Loading weights:  32%|███▏      | 140/443 [01:20<04:06,  1.23it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<04:05,  1.23it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<04:05,  1.23it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:58,  1.27it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:58,  1.27it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<03:58,  1.27it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  32%|███▏      | 140/443 [01:20<03:58,  1.27it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]        
Loading weights:  32%|███▏      | 140/443 [01:20<03:58,  1.27it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<03:57,  1.27it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<03:57,  1.27it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<04:08,  1.22it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<04:08,  1.22it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  31%|███▏      | 139/443 [01:20<04:08,  1.22it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
Loading weights:  32%|███▏      | 140/443 [01:20<04:07,  1.22it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]        
Loading weights:  32%|███▏      | 140/443 [01:20<04:07,  1.22it/s, Materializing param=model.layers.12.self_attn.k_norm.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<04:06,  1.22it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 141/443 [01:20<04:06,  1.22it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:58,  2.52it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:58,  2.52it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<03:29,  1.44it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<03:29,  1.44it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:58,  2.52it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<03:56,  1.27it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<03:56,  1.27it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<04:04,  1.23it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<04:04,  1.23it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<04:05,  1.22it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 142/443 [01:20<04:05,  1.22it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:59,  2.51it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:59,  2.51it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:59,  2.51it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:59,  2.50it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:59,  2.50it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:59,  2.50it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:59,  2.50it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:59,  2.50it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<02:04,  2.41it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<02:04,  2.41it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<02:04,  2.41it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<02:04,  2.41it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<02:04,  2.41it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<02:04,  2.41it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<02:04,  2.41it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<02:04,  2.41it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:55,  2.59it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:55,  2.59it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<01:55,  2.59it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:55,  2.59it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:55,  2.59it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:49,  2.74it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:49,  2.74it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:49,  2.74it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:48,  2.74it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:48,  2.74it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<02:03,  2.42it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<02:03,  2.42it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  32%|███▏      | 143/443 [01:20<02:03,  2.42it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<02:03,  2.42it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<02:03,  2.42it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 146/443 [01:20<01:48,  2.74it/s, Materializing param=model.layers.13.input_layernorm.weight] 
Loading weights:  33%|███▎      | 146/443 [01:20<01:48,  2.74it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  33%|███▎      | 147/443 [01:20<01:48,  2.74it/s, Materializing param=model.layers.13.mlp.down_proj.weight]  
Loading weights:  33%|███▎      | 147/443 [01:20<01:48,  2.74it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:50,  2.71it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:50,  2.71it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:50,  2.71it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:50,  2.71it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:50,  2.71it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:50,  2.72it/s, Materializing param=model.layers.12.self_attn.q_norm.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:50,  2.72it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 144/443 [01:20<01:50,  2.72it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:49,  2.72it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:49,  2.72it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 146/443 [01:20<01:49,  2.71it/s, Materializing param=model.layers.13.input_layernorm.weight] 
Loading weights:  33%|███▎      | 146/443 [01:20<01:49,  2.71it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  33%|███▎      | 147/443 [01:20<01:49,  2.71it/s, Materializing param=model.layers.13.mlp.down_proj.weight]  
Loading weights:  33%|███▎      | 147/443 [01:20<01:49,  2.71it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:35,  3.12it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:35,  3.12it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:35,  3.12it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 146/443 [01:20<01:49,  2.72it/s, Materializing param=model.layers.13.input_layernorm.weight] 
Loading weights:  33%|███▎      | 146/443 [01:20<01:49,  2.72it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  33%|███▎      | 147/443 [01:20<01:48,  2.72it/s, Materializing param=model.layers.13.mlp.down_proj.weight]  
Loading weights:  33%|███▎      | 147/443 [01:20<01:48,  2.72it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:35,  3.11it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:35,  3.11it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:35,  3.11it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 146/443 [01:20<01:35,  3.12it/s, Materializing param=model.layers.13.input_layernorm.weight] 
Loading weights:  33%|███▎      | 146/443 [01:20<01:35,  3.12it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  33%|███▎      | 147/443 [01:20<01:34,  3.12it/s, Materializing param=model.layers.13.mlp.down_proj.weight]  
Loading weights:  33%|███▎      | 147/443 [01:20<01:34,  3.12it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:38,  3.02it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:38,  3.02it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:38,  3.02it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 146/443 [01:20<01:35,  3.11it/s, Materializing param=model.layers.13.input_layernorm.weight] 
Loading weights:  33%|███▎      | 146/443 [01:20<01:35,  3.11it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  33%|███▎      | 147/443 [01:20<01:35,  3.11it/s, Materializing param=model.layers.13.mlp.down_proj.weight]  
Loading weights:  33%|███▎      | 147/443 [01:20<01:35,  3.11it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:33,  3.18it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:33,  3.18it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 145/443 [01:20<01:33,  3.18it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
Loading weights:  33%|███▎      | 146/443 [01:20<01:38,  3.02it/s, Materializing param=model.layers.13.input_layernorm.weight] 
Loading weights:  33%|███▎      | 146/443 [01:20<01:38,  3.02it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  33%|███▎      | 147/443 [01:20<01:37,  3.02it/s, Materializing param=model.layers.13.mlp.down_proj.weight]  
Loading weights:  33%|███▎      | 147/443 [01:20<01:37,  3.02it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 146/443 [01:20<01:33,  3.18it/s, Materializing param=model.layers.13.input_layernorm.weight] 
Loading weights:  33%|███▎      | 146/443 [01:20<01:33,  3.18it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  33%|███▎      | 147/443 [01:20<01:33,  3.18it/s, Materializing param=model.layers.13.mlp.down_proj.weight]  
Loading weights:  33%|███▎      | 147/443 [01:20<01:33,  3.18it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:20,  3.68it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 147/443 [01:21<01:58,  2.50it/s, Materializing param=model.layers.13.input_layernorm.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:21,  3.63it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 147/443 [01:21<01:58,  2.50it/s, Materializing param=model.layers.13.mlp.down_proj.weight]  
Loading weights:  33%|███▎      | 148/443 [01:21<01:20,  3.68it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:21,  3.63it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 147/443 [01:21<01:58,  2.50it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:20,  3.68it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:21,  3.63it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:58,  2.50it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:58,  2.50it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:24,  3.49it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:24,  3.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:24,  3.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:24,  3.49it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:24,  3.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:24,  3.49it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:28,  3.34it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:28,  3.34it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:28,  3.34it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:29,  3.31it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:29,  3.31it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:29,  3.31it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:25,  3.44it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:25,  3.44it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  33%|███▎      | 148/443 [01:21<01:25,  3.44it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:27,  1.99it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.17it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:18,  2.13it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight]  
Loading weights:  34%|███▎      | 149/443 [01:23<02:27,  1.99it/s, Materializing param=model.layers.13.mlp.up_proj.weight]  
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.17it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:27,  1.99it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:18,  2.13it/s, Materializing param=model.layers.13.mlp.up_proj.weight]  
Loading weights:  34%|███▎      | 149/443 [01:23<02:17,  2.14it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight]  
Loading weights:  34%|███▎      | 149/443 [01:23<02:18,  2.13it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.16it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.17it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:11,  2.24it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight]  
Loading weights:  34%|███▎      | 149/443 [01:23<02:17,  2.14it/s, Materializing param=model.layers.13.mlp.up_proj.weight]  
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.17it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:11,  2.24it/s, Materializing param=model.layers.13.mlp.up_proj.weight]  
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.16it/s, Materializing param=model.layers.13.mlp.up_proj.weight]  
Loading weights:  34%|███▎      | 149/443 [01:23<02:11,  2.24it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:17,  2.14it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▎      | 149/443 [01:23<02:15,  2.16it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:11,  2.24it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:11,  2.24it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 151/443 [01:23<02:10,  2.24it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]        
Loading weights:  34%|███▍      | 151/443 [01:23<02:10,  2.24it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:10,  2.24it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:10,  2.24it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:09,  2.24it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:09,  2.24it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:23<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:23<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:16,  2.14it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:16,  2.14it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:16,  2.14it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 151/443 [01:23<02:16,  2.14it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]        
Loading weights:  34%|███▍      | 151/443 [01:23<02:16,  2.14it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:15,  2.14it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:15,  2.14it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  35%|███▌      | 157/443 [01:23<01:17,  3.71it/s, Materializing param=model.layers.14.input_layernorm.weight] 
Loading weights:  35%|███▌      | 157/443 [01:23<01:17,  3.71it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  36%|███▌      | 158/443 [01:23<01:16,  3.71it/s, Materializing param=model.layers.14.mlp.down_proj.weight]  
Loading weights:  36%|███▌      | 158/443 [01:23<01:16,  3.71it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:15,  2.14it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:15,  2.14it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:14,  3.89it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:14,  3.89it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:14,  3.89it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:13,  3.89it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:13,  3.89it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:28,  1.97it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:30,  1.95it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:28,  1.97it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:28,  1.97it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 151/443 [01:23<02:28,  1.97it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]        
Loading weights:  34%|███▍      | 151/443 [01:23<02:28,  1.97it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:30,  1.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:30,  1.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 151/443 [01:23<02:29,  1.95it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]        
Loading weights:  34%|███▍      | 151/443 [01:23<02:29,  1.95it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:29,  1.95it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:29,  1.95it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:31,  1.93it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:31,  1.93it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:31,  1.93it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 151/443 [01:23<02:30,  1.93it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]        
Loading weights:  34%|███▍      | 151/443 [01:23<02:30,  1.93it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:30,  1.93it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:30,  1.93it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:29,  1.96it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:29,  1.96it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:29,  1.96it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 151/443 [01:23<02:28,  1.96it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]        
Loading weights:  34%|███▍      | 151/443 [01:23<02:28,  1.96it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:30,  1.95it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:30,  1.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:30,  1.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 151/443 [01:23<02:29,  1.95it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]        
Loading weights:  34%|███▍      | 151/443 [01:23<02:29,  1.95it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:38,  1.85it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:38,  1.85it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 150/443 [01:23<02:38,  1.85it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
Loading weights:  34%|███▍      | 151/443 [01:23<02:38,  1.85it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]        
Loading weights:  34%|███▍      | 151/443 [01:23<02:38,  1.85it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:37,  1.85it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:37,  1.85it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:37,  1.85it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:37,  1.85it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:28,  1.95it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:29,  1.93it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:29,  1.93it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:28,  1.95it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:27,  1.97it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:27,  1.97it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:27,  1.97it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:27,  1.97it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:29,  1.95it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<02:29,  1.95it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:28,  1.95it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<02:28,  1.95it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:23<00:59,  4.79it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:23<00:59,  4.79it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:23<00:59,  4.79it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 157/443 [01:23<00:59,  4.79it/s, Materializing param=model.layers.14.input_layernorm.weight] 
Loading weights:  35%|███▌      | 157/443 [01:23<00:59,  4.79it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  36%|███▌      | 158/443 [01:23<00:59,  4.79it/s, Materializing param=model.layers.14.mlp.down_proj.weight]  
Loading weights:  36%|███▌      | 158/443 [01:23<00:59,  4.79it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<01:46,  2.73it/s, Materializing param=model.layers.13.self_attn.k_norm.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<01:46,  2.73it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  34%|███▍      | 152/443 [01:23<01:46,  2.73it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<01:46,  2.73it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 153/443 [01:23<01:46,  2.73it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:46,  2.73it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:46,  2.73it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:45,  2.73it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:45,  2.73it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:22,  3.51it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:28,  3.25it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:22,  3.52it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:22,  3.51it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:28,  3.25it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:22,  3.52it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:28,  3.25it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:22,  3.52it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:28,  3.25it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:22,  3.51it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:28,  3.25it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:22,  3.51it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:22,  3.51it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:22,  3.51it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:21,  3.51it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:21,  3.51it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:21,  3.54it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:21,  3.54it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 154/443 [01:23<01:21,  3.54it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:21,  3.54it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▍      | 155/443 [01:23<01:21,  3.54it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:01,  4.67it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:01,  4.67it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:01,  4.67it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:06,  4.29it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:06,  4.29it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:06,  4.29it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▍      | 155/443 [01:24<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 155/443 [01:24<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▍      | 155/443 [01:24<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:17,  3.71it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:12,  3.94it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:12,  3.94it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:12,  3.94it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 157/443 [01:24<01:06,  4.29it/s, Materializing param=model.layers.14.input_layernorm.weight] 
Loading weights:  35%|███▌      | 157/443 [01:24<01:06,  4.29it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  36%|███▌      | 158/443 [01:24<01:06,  4.29it/s, Materializing param=model.layers.14.mlp.down_proj.weight]  
Loading weights:  36%|███▌      | 158/443 [01:24<01:06,  4.29it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  35%|███▌      | 157/443 [01:24<01:01,  4.67it/s, Materializing param=model.layers.14.input_layernorm.weight] 
Loading weights:  35%|███▌      | 157/443 [01:24<01:01,  4.67it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  36%|███▌      | 158/443 [01:24<01:01,  4.67it/s, Materializing param=model.layers.14.mlp.down_proj.weight]  
Loading weights:  36%|███▌      | 158/443 [01:24<01:01,  4.67it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  35%|███▌      | 157/443 [01:24<01:12,  3.94it/s, Materializing param=model.layers.14.input_layernorm.weight] 
Loading weights:  35%|███▌      | 157/443 [01:24<01:12,  3.94it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  35%|███▍      | 155/443 [01:24<01:19,  3.63it/s, Materializing param=model.layers.13.self_attn.q_norm.weight]
Loading weights:  35%|███▍      | 155/443 [01:24<01:19,  3.63it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▍      | 155/443 [01:24<01:19,  3.63it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:19,  3.63it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:19,  3.63it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 157/443 [01:24<01:17,  3.71it/s, Materializing param=model.layers.14.input_layernorm.weight] 
Loading weights:  35%|███▌      | 157/443 [01:24<01:17,  3.71it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  36%|███▌      | 158/443 [01:24<01:16,  3.71it/s, Materializing param=model.layers.14.mlp.down_proj.weight]  
Loading weights:  36%|███▌      | 158/443 [01:24<01:16,  3.71it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:08,  4.19it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:08,  4.19it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 156/443 [01:24<01:08,  4.19it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
Loading weights:  35%|███▌      | 157/443 [01:24<01:18,  3.63it/s, Materializing param=model.layers.14.input_layernorm.weight] 
Loading weights:  35%|███▌      | 157/443 [01:24<01:18,  3.63it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  35%|███▌      | 157/443 [01:24<01:08,  4.19it/s, Materializing param=model.layers.14.input_layernorm.weight] 
Loading weights:  35%|███▌      | 157/443 [01:24<01:08,  4.19it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  36%|███▌      | 158/443 [01:24<01:08,  4.19it/s, Materializing param=model.layers.14.mlp.down_proj.weight]  
Loading weights:  36%|███▌      | 158/443 [01:24<01:08,  4.19it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  36%|███▌      | 158/443 [01:24<01:18,  3.61it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:03,  4.47it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  36%|███▌      | 158/443 [01:24<01:11,  3.97it/s, Materializing param=model.layers.14.input_layernorm.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:05,  4.31it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:16,  3.73it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:11,  3.97it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:03,  4.51it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:04,  4.40it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:05,  4.31it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:16,  3.73it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:11,  3.97it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:05,  4.31it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:03,  4.47it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 158/443 [01:24<01:18,  3.61it/s, Materializing param=model.layers.14.mlp.down_proj.weight]  
Loading weights:  36%|███▌      | 159/443 [01:24<01:03,  4.51it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:11,  3.97it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:16,  3.73it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:04,  4.40it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 158/443 [01:24<01:18,  3.61it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:03,  4.47it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:03,  4.51it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:04,  4.40it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 158/443 [01:24<01:11,  3.97it/s, Materializing param=model.layers.14.mlp.down_proj.weight]  
Loading weights:  36%|███▌      | 159/443 [01:24<01:18,  3.61it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:18,  3.61it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 158/443 [01:24<01:11,  3.97it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:11,  3.97it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 159/443 [01:24<01:11,  3.97it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:10,  2.16it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:14,  2.10it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:10,  2.16it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:05,  2.26it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:05,  2.26it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:19,  2.02it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:13,  2.11it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:10,  2.16it/s, Materializing param=model.layers.14.mlp.up_proj.weight]  
Loading weights:  36%|███▌      | 160/443 [01:26<02:10,  2.16it/s, Materializing param=model.layers.14.mlp.up_proj.weight]  
Loading weights:  36%|███▌      | 160/443 [01:26<02:05,  2.26it/s, Materializing param=model.layers.14.mlp.up_proj.weight]  
Loading weights:  36%|███▌      | 160/443 [01:26<02:14,  2.10it/s, Materializing param=model.layers.14.mlp.up_proj.weight]  
Loading weights:  36%|███▌      | 160/443 [01:26<02:13,  2.11it/s, Materializing param=model.layers.14.mlp.up_proj.weight]  
Loading weights:  36%|███▌      | 160/443 [01:26<02:10,  2.16it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:10,  2.16it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:05,  2.26it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:19,  2.02it/s, Materializing param=model.layers.14.mlp.up_proj.weight]  
Loading weights:  36%|███▌      | 160/443 [01:26<02:14,  2.10it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:13,  2.11it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:19,  2.02it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:05,  2.26it/s, Materializing param=model.layers.14.mlp.up_proj.weight]  
Loading weights:  36%|███▌      | 160/443 [01:26<02:05,  2.26it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:09,  2.19it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
Loading weights:  36%|███▌      | 160/443 [01:26<02:09,  2.19it/s, Materializing param=model.layers.14.mlp.up_proj.weight]  
Loading weights:  36%|███▌      | 160/443 [01:26<02:09,  2.19it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:11,  1.47it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:23,  1.39it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:23,  1.39it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:23,  1.39it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:23,  1.39it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:11,  1.47it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:23,  1.39it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:17,  1.43it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:23,  1.39it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:11,  1.47it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:19,  1.42it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:17,  1.43it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:19,  1.42it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:19,  1.42it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<03:22,  1.39it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]        
Loading weights:  36%|███▋      | 161/443 [01:28<03:17,  1.43it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:17,  1.43it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<02:59,  1.57it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<03:11,  1.47it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]        
Loading weights:  37%|███▋      | 162/443 [01:28<03:22,  1.39it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:17,  1.43it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<02:59,  1.57it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<03:16,  1.43it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]        
Loading weights:  36%|███▋      | 161/443 [01:28<03:17,  1.43it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:28,  1.35it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<03:11,  1.47it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<02:59,  1.57it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<03:16,  1.43it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights:  36%|███▋      | 161/443 [01:28<03:28,  1.35it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<03:18,  1.42it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]        
Loading weights:  36%|███▋      | 161/443 [01:28<03:28,  1.35it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<02:59,  1.57it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]        
Loading weights:  37%|███▋      | 163/443 [01:28<03:15,  1.43it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<03:18,  1.42it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<03:22,  1.39it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<02:59,  1.57it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<03:15,  1.43it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<03:22,  1.39it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<03:17,  1.42it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<03:27,  1.35it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]        
Loading weights:  37%|███▋      | 162/443 [01:28<03:16,  1.43it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]        
Loading weights:  37%|███▋      | 162/443 [01:28<03:16,  1.43it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<03:17,  1.42it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<03:27,  1.35it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<02:58,  1.57it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<02:58,  1.57it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<03:15,  1.43it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<03:26,  1.35it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<03:26,  1.35it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<03:15,  1.43it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<02:49,  1.65it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
Loading weights:  37%|███▋      | 162/443 [01:28<02:49,  1.65it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]        
Loading weights:  37%|███▋      | 162/443 [01:28<02:49,  1.65it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<02:49,  1.65it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<02:49,  1.65it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:01,  2.29it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:01,  2.29it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:01,  2.31it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<01:59,  2.33it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:03,  2.27it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:01,  2.29it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<01:59,  2.33it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<02:18,  2.02it/s, Materializing param=model.layers.14.self_attn.k_norm.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:01,  2.31it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:03,  2.27it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<01:59,  2.33it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:01,  2.31it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:03,  2.27it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<02:18,  2.02it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 163/443 [01:28<02:18,  2.02it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:17,  2.02it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:05,  2.22it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:01,  2.29it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:17,  2.02it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:05,  2.22it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:05,  2.22it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:01,  2.29it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:01,  2.29it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:48,  1.65it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 164/443 [01:28<02:48,  1.65it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:28<02:48,  1.65it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 165/443 [01:28<02:48,  1.65it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 166/443 [01:28<02:47,  1.65it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 166/443 [01:28<02:47,  1.65it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:28<01:46,  2.62it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:28<01:46,  2.62it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 165/443 [01:28<01:46,  2.62it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 166/443 [01:28<01:45,  2.62it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 166/443 [01:28<01:45,  2.62it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:18,  3.53it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:18,  3.53it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:18,  3.53it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:50,  2.51it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:50,  2.51it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:50,  2.51it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<01:50,  2.51it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:44,  2.67it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<01:50,  2.51it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:51,  2.49it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:51,  2.49it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:51,  2.49it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:44,  2.67it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<01:51,  2.49it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<01:51,  2.49it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:44,  2.67it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<01:43,  2.67it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<01:43,  2.67it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:52,  2.48it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:51,  2.50it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:52,  2.48it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:51,  2.50it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:52,  2.48it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<01:51,  2.50it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<01:50,  2.50it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<01:50,  2.50it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<01:51,  2.48it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<01:51,  2.48it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:16,  3.61it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:16,  3.61it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:16,  3.61it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 168/443 [01:29<01:11,  3.85it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 168/443 [01:29<01:11,  3.85it/s, Materializing param=model.layers.15.input_layernorm.weight] 
Loading weights:  38%|███▊      | 168/443 [01:29<01:11,  3.85it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|███▊      | 169/443 [01:29<01:11,  3.85it/s, Materializing param=model.layers.15.mlp.down_proj.weight]  
Loading weights:  38%|███▊      | 169/443 [01:29<01:11,  3.85it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 168/443 [01:29<01:16,  3.61it/s, Materializing param=model.layers.15.input_layernorm.weight] 
Loading weights:  38%|███▊      | 168/443 [01:29<01:16,  3.61it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|███▊      | 169/443 [01:29<00:56,  4.83it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|███▊      | 169/443 [01:29<00:56,  4.83it/s, Materializing param=model.layers.15.mlp.down_proj.weight]  
Loading weights:  38%|███▊      | 169/443 [01:29<00:56,  4.83it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:23,  3.32it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:23,  3.32it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:23,  3.32it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<02:01,  2.29it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<02:01,  2.29it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 165/443 [01:29<02:01,  2.29it/s, Materializing param=model.layers.14.self_attn.q_norm.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<02:00,  2.29it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  37%|███▋      | 166/443 [01:29<02:00,  2.29it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<02:00,  2.29it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<02:00,  2.29it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:22,  3.35it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:22,  3.35it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:22,  3.35it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:22,  3.35it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:22,  3.35it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:22,  3.35it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:18,  3.52it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:22,  3.36it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:22,  3.36it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:18,  3.52it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:22,  3.36it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 167/443 [01:29<01:18,  3.52it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
Loading weights:  38%|███▊      | 168/443 [01:29<01:22,  3.32it/s, Materializing param=model.layers.15.input_layernorm.weight] 
Loading weights:  38%|███▊      | 168/443 [01:29<01:22,  3.32it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|███▊      | 169/443 [01:29<01:22,  3.32it/s, Materializing param=model.layers.15.mlp.down_proj.weight]  
Loading weights:  38%|███▊      | 169/443 [01:29<01:22,  3.32it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 168/443 [01:29<01:59,  2.29it/s, Materializing param=model.layers.15.input_layernorm.weight] 
Loading weights:  38%|███▊      | 168/443 [01:29<01:59,  2.29it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|███▊      | 169/443 [01:29<01:59,  2.29it/s, Materializing param=model.layers.15.mlp.down_proj.weight]  
Loading weights:  38%|███▊      | 169/443 [01:29<01:59,  2.29it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 168/443 [01:29<01:18,  3.52it/s, Materializing param=model.layers.15.input_layernorm.weight] 
Loading weights:  38%|███▊      | 168/443 [01:29<01:22,  3.35it/s, Materializing param=model.layers.15.input_layernorm.weight] 
Loading weights:  38%|███▊      | 168/443 [01:29<01:22,  3.35it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|███▊      | 169/443 [01:29<01:21,  3.35it/s, Materializing param=model.layers.15.mlp.down_proj.weight]  
Loading weights:  38%|███▊      | 169/443 [01:29<01:21,  3.35it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 168/443 [01:29<01:18,  3.52it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|███▊      | 169/443 [01:29<01:17,  3.52it/s, Materializing param=model.layers.15.mlp.down_proj.weight]  
Loading weights:  38%|███▊      | 169/443 [01:29<01:17,  3.52it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 168/443 [01:29<01:22,  3.35it/s, Materializing param=model.layers.15.input_layernorm.weight] 
Loading weights:  38%|███▊      | 168/443 [01:29<01:21,  3.36it/s, Materializing param=model.layers.15.input_layernorm.weight] 
Loading weights:  38%|███▊      | 168/443 [01:29<01:22,  3.35it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|███▊      | 168/443 [01:29<01:21,  3.36it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|███▊      | 169/443 [01:29<01:21,  3.35it/s, Materializing param=model.layers.15.mlp.down_proj.weight]  
Loading weights:  38%|███▊      | 169/443 [01:29<01:21,  3.35it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 169/443 [01:30<02:10,  2.10it/s, Materializing param=model.layers.15.input_layernorm.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:52,  2.43it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:49,  2.49it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:52,  2.42it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:52,  2.42it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<00:56,  4.83it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<02:00,  2.26it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:46,  2.57it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:52,  2.43it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:52,  2.42it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:52,  2.43it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:52,  2.42it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:46,  2.57it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<00:56,  4.83it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:52,  2.42it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<02:00,  2.26it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 169/443 [01:30<02:10,  2.10it/s, Materializing param=model.layers.15.mlp.down_proj.weight]  
Loading weights:  38%|███▊      | 170/443 [01:30<01:52,  2.42it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:49,  2.49it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:46,  2.57it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 169/443 [01:30<02:10,  2.10it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<02:00,  2.26it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<01:49,  2.49it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<02:10,  2.10it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  38%|███▊      | 170/443 [01:30<02:10,  2.10it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<02:59,  1.52it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<03:01,  1.50it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<02:53,  1.57it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<02:58,  1.53it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<02:44,  1.65it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<03:17,  1.37it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<03:12,  1.41it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<03:02,  1.49it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<02:59,  1.52it/s, Materializing param=model.layers.15.mlp.up_proj.weight]  
Loading weights:  39%|███▊      | 171/443 [01:32<03:01,  1.50it/s, Materializing param=model.layers.15.mlp.up_proj.weight]  
Loading weights:  39%|███▊      | 171/443 [01:32<02:53,  1.57it/s, Materializing param=model.layers.15.mlp.up_proj.weight]  
Loading weights:  39%|███▊      | 171/443 [01:32<02:59,  1.52it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<02:44,  1.65it/s, Materializing param=model.layers.15.mlp.up_proj.weight]  
Loading weights:  39%|███▊      | 171/443 [01:32<03:17,  1.37it/s, Materializing param=model.layers.15.mlp.up_proj.weight]  
Loading weights:  39%|███▊      | 171/443 [01:32<03:01,  1.50it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<03:12,  1.41it/s, Materializing param=model.layers.15.mlp.up_proj.weight]  
Loading weights:  39%|███▊      | 171/443 [01:32<03:02,  1.49it/s, Materializing param=model.layers.15.mlp.up_proj.weight]  
Loading weights:  39%|███▊      | 171/443 [01:32<02:53,  1.57it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<02:44,  1.65it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<03:17,  1.37it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<03:12,  1.41it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<03:02,  1.49it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▊      | 171/443 [01:32<02:58,  1.53it/s, Materializing param=model.layers.15.mlp.up_proj.weight]  
Loading weights:  39%|███▊      | 171/443 [01:32<02:58,  1.53it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:51,  1.17it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<04:07,  1.10it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:56,  1.14it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<04:07,  1.10it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:51,  1.17it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:56,  1.14it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<04:07,  1.10it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:56,  1.14it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:51,  1.17it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<04:06,  1.10it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<04:06,  1.10it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<03:50,  1.17it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]        
Loading weights:  39%|███▉      | 172/443 [01:34<04:06,  1.10it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<03:50,  1.17it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<04:05,  1.10it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]        
Loading weights:  39%|███▉      | 173/443 [01:34<04:06,  1.10it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]        
Loading weights:  39%|███▉      | 173/443 [01:34<04:05,  1.10it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:47,  1.19it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<03:55,  1.14it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]        
Loading weights:  39%|███▉      | 172/443 [01:34<03:47,  1.19it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<04:06,  1.10it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:51,  1.17it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:55,  1.15it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:47,  1.19it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<03:55,  1.14it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:33,  1.27it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:55,  1.15it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:51,  1.17it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:55,  1.15it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<03:55,  1.14it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<03:46,  1.19it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]        
Loading weights:  39%|███▉      | 172/443 [01:34<03:33,  1.27it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<03:55,  1.14it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<03:46,  1.19it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:51,  1.17it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 172/443 [01:34<03:33,  1.27it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<04:05,  1.10it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<03:45,  1.19it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<03:54,  1.15it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]        
Loading weights:  39%|███▉      | 174/443 [01:34<04:05,  1.10it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<03:45,  1.19it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<03:50,  1.17it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]        
Loading weights:  39%|███▉      | 173/443 [01:34<03:54,  1.15it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<03:50,  1.17it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<03:54,  1.15it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<03:50,  1.17it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<03:54,  1.15it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<03:50,  1.17it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 173/443 [01:34<03:33,  1.27it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]        
Loading weights:  39%|███▉      | 173/443 [01:34<03:33,  1.27it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<03:32,  1.27it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<03:32,  1.27it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<02:46,  1.61it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<02:46,  1.61it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<02:46,  1.61it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:45,  1.61it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:45,  1.61it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:13,  2.01it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:17,  1.95it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<02:35,  1.73it/s, Materializing param=model.layers.15.self_attn.k_norm.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:13,  2.01it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:09,  2.08it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:13,  2.01it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:13,  2.01it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:13,  2.01it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:13,  2.01it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:17,  1.95it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:09,  2.08it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:17,  1.95it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:09,  2.08it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<02:35,  1.73it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  39%|███▉      | 174/443 [01:34<02:35,  1.73it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:35,  1.73it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:35,  1.73it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:10,  2.05it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:13,  2.00it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:10,  2.05it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:13,  2.00it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:10,  2.05it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 175/443 [01:34<02:13,  2.00it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:12,  2.01it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:16,  1.95it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:12,  2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:16,  1.95it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:12,  2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:12,  2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:12,  2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:13,  2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:05,  2.13it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:10,  2.05it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:13,  2.00it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:52,  2.36it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:13,  2.01it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:54,  2.32it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:10,  2.05it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:09,  2.07it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:13,  2.00it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:52,  2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:54,  2.32it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:52,  2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:52,  2.36it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:09,  2.07it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:50,  2.40it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:52,  2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:09,  2.07it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:50,  2.40it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:52,  2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:52,  2.36it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:54,  2.32it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:50,  2.40it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:05,  2.13it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:52,  2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<02:12,  2.01it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<02:08,  2.07it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 176/443 [01:35<02:05,  2.13it/s, Materializing param=model.layers.15.self_attn.q_norm.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<01:52,  2.36it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<02:08,  2.07it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<02:12,  2.01it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<02:04,  2.13it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|███▉      | 177/443 [01:35<02:04,  2.13it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:55,  2.30it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:47,  2.46it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:45,  2.52it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:56,  2.27it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:53,  2.33it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:54,  2.30it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:47,  2.46it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:55,  2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:50,  2.40it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:55,  2.30it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:56,  2.27it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:55,  2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:54,  2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:53,  2.33it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:47,  2.46it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:50,  2.40it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:54,  2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:55,  2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:56,  2.27it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:55,  2.30it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:50,  2.40it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:45,  2.52it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:45,  2.52it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 178/443 [01:35<01:53,  2.33it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 179/443 [01:35<01:56,  2.27it/s, Materializing param=model.layers.16.input_layernorm.weight] 
Loading weights:  40%|████      | 179/443 [01:35<01:49,  2.40it/s, Materializing param=model.layers.16.input_layernorm.weight] 
Loading weights:  40%|████      | 179/443 [01:35<01:44,  2.52it/s, Materializing param=model.layers.16.input_layernorm.weight] 
Loading weights:  40%|████      | 179/443 [01:35<01:54,  2.30it/s, Materializing param=model.layers.16.input_layernorm.weight] 
Loading weights:  40%|████      | 179/443 [01:35<01:47,  2.46it/s, Materializing param=model.layers.16.input_layernorm.weight] 
Loading weights:  40%|████      | 179/443 [01:35<01:47,  2.46it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  40%|████      | 179/443 [01:35<01:54,  2.30it/s, Materializing param=model.layers.16.input_layernorm.weight] 
Loading weights:  40%|████      | 179/443 [01:35<01:54,  2.30it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  40%|████      | 179/443 [01:35<01:54,  2.30it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  40%|████      | 179/443 [01:35<01:44,  2.52it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  40%|████      | 179/443 [01:35<01:35,  2.75it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  40%|████      | 179/443 [01:35<01:49,  2.40it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  40%|████      | 179/443 [01:35<01:56,  2.27it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  41%|████      | 180/443 [01:35<01:20,  3.27it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  40%|████      | 179/443 [01:35<01:35,  2.75it/s, Materializing param=model.layers.16.input_layernorm.weight] 
Loading weights:  40%|████      | 179/443 [01:35<01:35,  2.75it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  41%|████      | 180/443 [01:35<01:20,  3.27it/s, Materializing param=model.layers.16.mlp.down_proj.weight]  
Loading weights:  41%|████      | 180/443 [01:35<01:20,  3.27it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  40%|████      | 179/443 [01:35<01:37,  2.72it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
Loading weights:  41%|████      | 180/443 [01:35<01:19,  3.31it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  41%|████      | 180/443 [01:35<01:21,  3.23it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  41%|████      | 180/443 [01:35<01:35,  2.75it/s, Materializing param=model.layers.16.mlp.down_proj.weight]  
Loading weights:  41%|████      | 180/443 [01:35<01:19,  3.31it/s, Materializing param=model.layers.16.mlp.down_proj.weight]  
Loading weights:  41%|████      | 180/443 [01:35<01:35,  2.75it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  40%|████      | 179/443 [01:35<01:37,  2.72it/s, Materializing param=model.layers.16.input_layernorm.weight] 
Loading weights:  41%|████      | 180/443 [01:35<01:21,  3.23it/s, Materializing param=model.layers.16.mlp.down_proj.weight]  
Loading weights:  41%|████      | 180/443 [01:35<01:19,  3.31it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  41%|████      | 180/443 [01:35<01:21,  3.23it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  40%|████      | 179/443 [01:35<01:37,  2.72it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  41%|████      | 180/443 [01:35<01:36,  2.72it/s, Materializing param=model.layers.16.mlp.down_proj.weight]  
Loading weights:  41%|████      | 180/443 [01:35<01:36,  2.72it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<01:20,  3.27it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<01:19,  3.31it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<01:21,  3.23it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<01:20,  3.27it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<01:19,  3.31it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<02:32,  1.72it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  41%|████      | 180/443 [01:37<02:35,  1.69it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  41%|████      | 181/443 [01:37<02:32,  1.71it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<01:21,  3.23it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<02:32,  1.72it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<02:32,  1.71it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<02:32,  1.72it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<02:32,  1.71it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 180/443 [01:37<02:36,  1.68it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  41%|████      | 180/443 [01:37<02:36,  1.68it/s, Materializing param=model.layers.16.mlp.down_proj.weight]  
Loading weights:  41%|████      | 180/443 [01:37<02:36,  1.68it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  41%|████      | 180/443 [01:37<02:35,  1.69it/s, Materializing param=model.layers.16.mlp.down_proj.weight]  
Loading weights:  41%|████      | 181/443 [01:37<02:36,  1.68it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 180/443 [01:37<02:35,  1.69it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<02:36,  1.68it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<02:34,  1.69it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:37<02:34,  1.69it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 180/443 [01:38<02:56,  1.49it/s, Materializing param=model.layers.16.input_layernorm.weight]
Loading weights:  41%|████      | 180/443 [01:38<02:56,  1.49it/s, Materializing param=model.layers.16.mlp.down_proj.weight]  
Loading weights:  41%|████      | 180/443 [01:38<02:56,  1.49it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
Loading weights:  41%|████      | 181/443 [01:38<02:55,  1.49it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 181/443 [01:38<02:55,  1.49it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:53,  1.12it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:20,  1.30it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<04:04,  1.07it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:45,  1.16it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<04:05,  1.06it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:25,  1.27it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:54,  1.11it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:20,  1.30it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:53,  1.12it/s, Materializing param=model.layers.16.mlp.up_proj.weight]  
Loading weights:  41%|████      | 182/443 [01:40<04:04,  1.07it/s, Materializing param=model.layers.16.mlp.up_proj.weight]  
Loading weights:  41%|████      | 182/443 [01:40<04:05,  1.06it/s, Materializing param=model.layers.16.mlp.up_proj.weight]  
Loading weights:  41%|████      | 182/443 [01:40<03:53,  1.12it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<04:04,  1.07it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<04:05,  1.06it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:45,  1.16it/s, Materializing param=model.layers.16.mlp.up_proj.weight]  
Loading weights:  41%|████      | 182/443 [01:40<03:20,  1.30it/s, Materializing param=model.layers.16.mlp.up_proj.weight]  
Loading weights:  41%|████      | 182/443 [01:40<03:54,  1.11it/s, Materializing param=model.layers.16.mlp.up_proj.weight]  
Loading weights:  41%|████      | 182/443 [01:40<03:25,  1.27it/s, Materializing param=model.layers.16.mlp.up_proj.weight]  
Loading weights:  41%|████      | 182/443 [01:40<03:45,  1.16it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:25,  1.27it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:54,  1.11it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:20,  1.30it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████      | 182/443 [01:40<03:20,  1.30it/s, Materializing param=model.layers.16.mlp.up_proj.weight]  
Loading weights:  41%|████      | 182/443 [01:40<03:20,  1.30it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:24,  1.27it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:24,  1.27it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  42%|████▏     | 184/443 [01:40<03:24,  1.27it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]        
Loading weights:  42%|████▏     | 184/443 [01:40<03:24,  1.27it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:23,  1.27it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:23,  1.27it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:40<03:22,  1.27it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:40<03:22,  1.27it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:46,  2.41it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:46,  2.41it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:46,  2.41it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 188/443 [01:40<01:45,  2.41it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  42%|████▏     | 188/443 [01:40<01:45,  2.41it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:39,  1.19it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:39,  1.19it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:39,  1.19it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  42%|████▏     | 184/443 [01:40<03:38,  1.19it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]        
Loading weights:  42%|████▏     | 184/443 [01:40<03:38,  1.19it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:35,  1.20it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:35,  1.20it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:35,  1.20it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  42%|████▏     | 184/443 [01:40<03:35,  1.20it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]        
Loading weights:  42%|████▏     | 184/443 [01:40<03:35,  1.20it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:34,  1.20it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:34,  1.20it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:37,  1.19it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:37,  1.19it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:40<03:36,  1.19it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:40<03:36,  1.19it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:31,  1.23it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:31,  1.23it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:31,  1.23it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  42%|████▏     | 184/443 [01:40<03:30,  1.23it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]        
Loading weights:  42%|████▏     | 184/443 [01:40<03:30,  1.23it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights:  42%|████▏     | 186/443 [01:40<03:33,  1.20it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:40<03:33,  1.20it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:29,  1.23it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:29,  1.23it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:40<03:29,  1.23it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:40<03:29,  1.23it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:41,  2.52it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:41,  2.52it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:41,  2.52it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 188/443 [01:40<01:41,  2.52it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  42%|████▏     | 188/443 [01:40<01:41,  2.52it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:40<01:29,  2.84it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:40<01:29,  2.84it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:40<01:29,  2.84it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:50,  2.33it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:50,  2.33it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:50,  2.33it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:49,  2.35it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:49,  2.35it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 187/443 [01:40<01:49,  2.35it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 188/443 [01:40<01:48,  2.35it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  42%|████▏     | 188/443 [01:40<01:48,  2.35it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  43%|████▎     | 190/443 [01:40<01:28,  2.84it/s, Materializing param=model.layers.17.input_layernorm.weight] 
Loading weights:  43%|████▎     | 190/443 [01:40<01:28,  2.84it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:58,  1.09it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:58,  1.09it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:58,  1.09it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  42%|████▏     | 184/443 [01:40<03:57,  1.09it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]        
Loading weights:  42%|████▏     | 184/443 [01:40<03:57,  1.09it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:56,  1.09it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:56,  1.09it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:49,  1.13it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:23,  1.28it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:23,  1.28it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:23,  1.28it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:49,  1.13it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  42%|████▏     | 184/443 [01:40<03:22,  1.28it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]        
Loading weights:  41%|████▏     | 183/443 [01:40<03:49,  1.13it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  42%|████▏     | 184/443 [01:40<03:22,  1.28it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights:  42%|████▏     | 184/443 [01:40<03:48,  1.13it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]        
Loading weights:  42%|████▏     | 184/443 [01:40<03:48,  1.13it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:22,  1.28it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:22,  1.28it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:48,  1.13it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:48,  1.13it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:23,  1.28it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:23,  1.28it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  41%|████▏     | 183/443 [01:40<03:23,  1.28it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
Loading weights:  42%|████▏     | 184/443 [01:40<03:22,  1.28it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]        
Loading weights:  42%|████▏     | 184/443 [01:40<03:22,  1.28it/s, Materializing param=model.layers.16.self_attn.k_norm.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:22,  1.28it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 185/443 [01:40<03:22,  1.28it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:40<03:56,  1.09it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:40<03:56,  1.09it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:41<03:47,  1.13it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:41<03:47,  1.13it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:41<03:21,  1.28it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:41<03:21,  1.28it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:37,  2.61it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:37,  2.61it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:37,  2.61it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:37,  2.61it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:37,  2.61it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:41<03:21,  1.28it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 186/443 [01:41<03:21,  1.28it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:23,  3.04it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:23,  3.04it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:23,  3.04it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 190/443 [01:41<01:36,  2.61it/s, Materializing param=model.layers.17.input_layernorm.weight] 
Loading weights:  43%|████▎     | 190/443 [01:41<01:36,  2.61it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  43%|████▎     | 190/443 [01:41<01:23,  3.04it/s, Materializing param=model.layers.17.input_layernorm.weight] 
Loading weights:  43%|████▎     | 190/443 [01:41<01:23,  3.04it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  43%|████▎     | 191/443 [01:41<01:36,  2.61it/s, Materializing param=model.layers.17.mlp.down_proj.weight]  
Loading weights:  43%|████▎     | 191/443 [01:41<01:36,  2.61it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 191/443 [01:41<01:22,  3.04it/s, Materializing param=model.layers.17.mlp.down_proj.weight]  
Loading weights:  43%|████▎     | 191/443 [01:41<01:22,  3.04it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:20,  3.17it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:20,  3.17it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:20,  3.17it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 190/443 [01:41<01:19,  3.17it/s, Materializing param=model.layers.17.input_layernorm.weight] 
Loading weights:  43%|████▎     | 190/443 [01:41<01:19,  3.17it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  43%|████▎     | 191/443 [01:41<01:19,  3.17it/s, Materializing param=model.layers.17.mlp.down_proj.weight]  
Loading weights:  43%|████▎     | 191/443 [01:41<01:19,  3.17it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:49,  2.34it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:49,  2.34it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:49,  2.34it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:45,  2.44it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:45,  2.44it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:45,  2.44it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:55,  2.21it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:55,  2.21it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:55,  2.21it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:55,  2.21it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:55,  2.21it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:44,  2.45it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:44,  2.45it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 187/443 [01:41<01:44,  2.45it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:44,  2.45it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:44,  2.45it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:43,  2.45it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:54,  2.21it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:54,  2.21it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:43,  2.45it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:32,  2.75it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:32,  2.75it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:32,  2.75it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:32,  2.75it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:32,  2.75it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 190/443 [01:41<01:16,  3.29it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 190/443 [01:41<01:16,  3.29it/s, Materializing param=model.layers.17.input_layernorm.weight] 
Loading weights:  43%|████▎     | 190/443 [01:41<01:16,  3.29it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  43%|████▎     | 191/443 [01:41<01:16,  3.29it/s, Materializing param=model.layers.17.mlp.down_proj.weight]  
Loading weights:  43%|████▎     | 191/443 [01:41<01:16,  3.29it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 190/443 [01:41<01:10,  3.61it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 190/443 [01:41<01:10,  3.61it/s, Materializing param=model.layers.17.input_layernorm.weight] 
Loading weights:  43%|████▎     | 190/443 [01:41<01:10,  3.61it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  43%|████▎     | 191/443 [01:41<01:09,  3.61it/s, Materializing param=model.layers.17.mlp.down_proj.weight]  
Loading weights:  43%|████▎     | 191/443 [01:41<01:09,  3.61it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:35,  2.67it/s, Materializing param=model.layers.16.self_attn.q_norm.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:35,  2.67it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  42%|████▏     | 188/443 [01:41<01:35,  2.67it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:35,  2.67it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 189/443 [01:41<01:35,  2.67it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
Loading weights:  43%|████▎     | 190/443 [01:41<01:34,  2.67it/s, Materializing param=model.layers.17.input_layernorm.weight] 
Loading weights:  43%|████▎     | 190/443 [01:41<01:34,  2.67it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  43%|████▎     | 191/443 [01:41<01:34,  2.67it/s, Materializing param=model.layers.17.mlp.down_proj.weight]  
Loading weights:  43%|████▎     | 191/443 [01:41<01:34,  2.67it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 190/443 [01:41<01:31,  2.75it/s, Materializing param=model.layers.17.input_layernorm.weight] 
Loading weights:  43%|████▎     | 190/443 [01:41<01:31,  2.75it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  43%|████▎     | 191/443 [01:41<01:31,  2.75it/s, Materializing param=model.layers.17.mlp.down_proj.weight]  
Loading weights:  43%|████▎     | 191/443 [01:41<01:31,  2.75it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 191/443 [01:42<01:54,  2.19it/s, Materializing param=model.layers.17.input_layernorm.weight]
Loading weights:  43%|████▎     | 191/443 [01:42<01:54,  2.19it/s, Materializing param=model.layers.17.mlp.down_proj.weight]  
Loading weights:  43%|████▎     | 191/443 [01:42<01:54,  2.19it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:27,  2.86it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:34,  2.66it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:22,  3.04it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:31,  2.75it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:31,  2.74it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:21,  3.06it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:34,  2.66it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:22,  3.04it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:31,  2.75it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:22,  3.04it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:31,  2.74it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:31,  2.75it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:21,  3.06it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:34,  2.66it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:31,  2.74it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:21,  3.06it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:27,  2.86it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:54,  2.19it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:54,  2.19it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:27,  2.86it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:40,  2.51it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:40,  2.51it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  43%|████▎     | 192/443 [01:42<01:40,  2.51it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<02:35,  1.61it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<02:31,  1.65it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<02:35,  1.61it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<01:27,  2.86it/s, Materializing param=model.layers.17.mlp.up_proj.weight]  
Loading weights:  44%|████▎     | 193/443 [01:44<02:26,  1.71it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<02:31,  1.65it/s, Materializing param=model.layers.17.mlp.up_proj.weight]  
Loading weights:  44%|████▎     | 193/443 [01:44<02:23,  1.74it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<02:32,  1.64it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<02:35,  1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight]  
Loading weights:  44%|████▎     | 193/443 [01:44<02:35,  1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight]  
Loading weights:  44%|████▎     | 193/443 [01:44<02:26,  1.71it/s, Materializing param=model.layers.17.mlp.up_proj.weight]  
Loading weights:  44%|████▎     | 193/443 [01:44<02:31,  1.65it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<02:35,  1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<02:26,  1.71it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<02:35,  1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<02:32,  1.64it/s, Materializing param=model.layers.17.mlp.up_proj.weight]  
Loading weights:  44%|████▎     | 193/443 [01:44<02:23,  1.74it/s, Materializing param=model.layers.17.mlp.up_proj.weight]  
Loading weights:  44%|████▎     | 193/443 [01:44<02:23,  1.74it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<01:27,  2.86it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<02:32,  1.64it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▎     | 193/443 [01:44<01:39,  2.51it/s, Materializing param=model.layers.17.mlp.up_proj.weight]  
Loading weights:  44%|████▎     | 193/443 [01:44<01:39,  2.51it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:44<02:13,  1.86it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:44<02:13,  1.86it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 194/443 [01:44<02:13,  1.86it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 195/443 [01:44<02:12,  1.86it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]        
Loading weights:  44%|████▍     | 195/443 [01:44<02:12,  1.86it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights:  44%|████▍     | 196/443 [01:44<02:12,  1.86it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 196/443 [01:44<02:12,  1.86it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:44<01:28,  2.78it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:44<01:28,  2.78it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:44<01:28,  2.78it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:31,  1.65it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:31,  1.65it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:31,  1.65it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 195/443 [01:45<02:30,  1.65it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]        
Loading weights:  44%|████▍     | 195/443 [01:45<02:30,  1.65it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:29,  1.65it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:29,  1.65it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:34,  1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:34,  1.61it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:34,  1.61it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 195/443 [01:45<02:33,  1.61it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]        
Loading weights:  44%|████▍     | 195/443 [01:45<02:33,  1.61it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:33,  1.61it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:33,  1.61it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:24,  1.72it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:24,  1.72it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:24,  1.72it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 195/443 [01:45<02:24,  1.72it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]        
Loading weights:  44%|████▍     | 195/443 [01:45<02:24,  1.72it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:23,  1.72it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:23,  1.72it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:28,  1.68it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:28,  1.68it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:28,  1.68it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 195/443 [01:45<02:27,  1.68it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]        
Loading weights:  44%|████▍     | 195/443 [01:45<02:27,  1.68it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:27,  1.68it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:27,  1.68it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:32,  1.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:32,  1.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:29,  1.65it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:29,  1.65it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:33,  1.63it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:33,  1.63it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:33,  1.63it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 195/443 [01:45<02:32,  1.63it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]        
Loading weights:  44%|████▍     | 195/443 [01:45<02:32,  1.63it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:31,  1.63it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:31,  1.63it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:23,  1.72it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:23,  1.72it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:36,  1.59it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:36,  1.59it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:36,  1.59it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 195/443 [01:45<02:36,  1.59it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]        
Loading weights:  44%|████▍     | 195/443 [01:45<02:36,  1.59it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:26,  1.68it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:26,  1.68it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:34,  1.61it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:34,  1.61it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 194/443 [01:45<02:34,  1.61it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
Loading weights:  44%|████▍     | 195/443 [01:45<02:33,  1.61it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]        
Loading weights:  44%|████▍     | 195/443 [01:45<02:33,  1.61it/s, Materializing param=model.layers.17.self_attn.k_norm.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:32,  1.61it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:32,  1.61it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:31,  1.63it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:31,  1.63it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:35,  1.59it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 196/443 [01:45<02:35,  1.59it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:35,  1.59it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:35,  1.59it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:32,  1.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  44%|████▍     | 197/443 [01:45<02:32,  1.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:32,  2.65it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:31,  2.67it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:32,  2.65it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:39,  2.47it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:31,  2.67it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:32,  2.65it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:31,  2.67it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:28,  2.76it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:31,  2.68it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:28,  2.76it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:33,  2.61it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:31,  2.67it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:31,  2.68it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:33,  2.61it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:28,  2.77it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:31,  2.67it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:31,  2.68it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:33,  2.61it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:28,  2.76it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:32,  2.65it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:28,  2.77it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:39,  2.47it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:28,  2.77it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:32,  2.65it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:33,  2.61it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:31,  2.68it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:33,  2.61it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:28,  2.76it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:31,  2.68it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:39,  2.47it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:28,  2.76it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:28,  2.77it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:28,  2.77it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:38,  2.47it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:38,  2.47it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:28,  2.78it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 198/443 [01:45<01:28,  2.78it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:36,  2.52it/s, Materializing param=model.layers.17.self_attn.q_norm.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:36,  2.52it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▍     | 199/443 [01:45<01:36,  2.52it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:36,  2.52it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:36,  2.52it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 201/443 [01:45<01:14,  3.27it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 201/443 [01:45<01:14,  3.27it/s, Materializing param=model.layers.18.input_layernorm.weight] 
Loading weights:  45%|████▌     | 201/443 [01:45<01:14,  3.27it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  46%|████▌     | 202/443 [01:45<01:13,  3.27it/s, Materializing param=model.layers.18.mlp.down_proj.weight]  
Loading weights:  46%|████▌     | 202/443 [01:45<01:13,  3.27it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:15,  3.24it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:13,  3.31it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:13,  3.31it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:15,  3.24it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:13,  3.31it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:15,  3.24it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:10,  3.43it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:10,  3.43it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:10,  3.43it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:20,  3.01it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:20,  3.01it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:20,  3.01it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:13,  3.32it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:13,  3.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:13,  3.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:13,  3.32it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:13,  3.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:13,  3.32it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:11,  3.40it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:11,  3.40it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 200/443 [01:45<01:11,  3.40it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 201/443 [01:45<01:14,  3.24it/s, Materializing param=model.layers.18.input_layernorm.weight] 
Loading weights:  45%|████▌     | 201/443 [01:45<01:14,  3.24it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  46%|████▌     | 202/443 [01:45<01:14,  3.24it/s, Materializing param=model.layers.18.mlp.down_proj.weight]  
Loading weights:  46%|████▌     | 202/443 [01:45<01:14,  3.24it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  45%|████▌     | 201/443 [01:45<01:12,  3.32it/s, Materializing param=model.layers.18.input_layernorm.weight] 
Loading weights:  45%|████▌     | 201/443 [01:45<01:12,  3.32it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  46%|████▌     | 202/443 [01:45<01:12,  3.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight]  
Loading weights:  46%|████▌     | 202/443 [01:45<01:12,  3.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  45%|████▌     | 201/443 [01:45<01:10,  3.43it/s, Materializing param=model.layers.18.input_layernorm.weight] 
Loading weights:  45%|████▌     | 201/443 [01:45<01:10,  3.43it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  45%|████▌     | 201/443 [01:45<01:20,  3.01it/s, Materializing param=model.layers.18.input_layernorm.weight] 
Loading weights:  46%|████▌     | 202/443 [01:45<01:10,  3.43it/s, Materializing param=model.layers.18.mlp.down_proj.weight]  
Loading weights:  46%|████▌     | 202/443 [01:45<01:10,  3.43it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  45%|████▌     | 201/443 [01:45<01:20,  3.01it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  45%|████▌     | 201/443 [01:45<01:12,  3.32it/s, Materializing param=model.layers.18.input_layernorm.weight] 
Loading weights:  45%|████▌     | 201/443 [01:45<01:12,  3.32it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  46%|████▌     | 202/443 [01:45<01:12,  3.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight]  
Loading weights:  46%|████▌     | 202/443 [01:45<01:12,  3.32it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  46%|████▌     | 202/443 [01:47<01:51,  2.16it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:52,  2.14it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:34,  2.55it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:35,  2.51it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:35,  2.52it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:52,  2.14it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 202/443 [01:47<01:51,  2.16it/s, Materializing param=model.layers.18.mlp.down_proj.weight]  
Loading weights:  46%|████▌     | 203/443 [01:47<01:36,  2.49it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:35,  2.51it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:35,  2.52it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:52,  2.14it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:34,  2.55it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:35,  2.51it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:35,  2.52it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:34,  2.55it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:36,  2.49it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 202/443 [01:47<01:51,  2.16it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:36,  2.49it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:51,  2.16it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<01:51,  2.16it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  45%|████▌     | 201/443 [01:47<02:07,  1.90it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 201/443 [01:47<02:07,  1.90it/s, Materializing param=model.layers.18.input_layernorm.weight] 
Loading weights:  45%|████▌     | 201/443 [01:47<02:04,  1.94it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
Loading weights:  45%|████▌     | 201/443 [01:47<02:07,  1.90it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  45%|████▌     | 201/443 [01:47<02:04,  1.94it/s, Materializing param=model.layers.18.input_layernorm.weight] 
Loading weights:  45%|████▌     | 201/443 [01:47<02:04,  1.94it/s, Materializing param=model.layers.18.input_layernorm.weight]
Loading weights:  46%|████▌     | 202/443 [01:47<02:07,  1.90it/s, Materializing param=model.layers.18.mlp.down_proj.weight]  
Loading weights:  46%|████▌     | 202/443 [01:47<02:07,  1.90it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  46%|████▌     | 202/443 [01:47<02:03,  1.94it/s, Materializing param=model.layers.18.mlp.down_proj.weight]  
Loading weights:  46%|████▌     | 203/443 [01:47<02:06,  1.90it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 202/443 [01:47<02:03,  1.94it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<02:06,  1.90it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<02:03,  1.94it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 203/443 [01:47<02:03,  1.94it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:32,  1.56it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:32,  1.57it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:32,  1.56it/s, Materializing param=model.layers.18.mlp.up_proj.weight]  
Loading weights:  46%|████▌     | 204/443 [01:49<02:20,  1.70it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:32,  1.56it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:20,  1.70it/s, Materializing param=model.layers.18.mlp.up_proj.weight]  
Loading weights:  46%|████▌     | 204/443 [01:49<02:25,  1.64it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:32,  1.57it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:31,  1.58it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:32,  1.57it/s, Materializing param=model.layers.18.mlp.up_proj.weight]  
Loading weights:  46%|████▌     | 204/443 [01:49<02:20,  1.70it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:18,  1.73it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:32,  1.57it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:32,  1.57it/s, Materializing param=model.layers.18.mlp.up_proj.weight]  
Loading weights:  46%|████▌     | 204/443 [01:49<02:31,  1.58it/s, Materializing param=model.layers.18.mlp.up_proj.weight]  
Loading weights:  46%|████▌     | 204/443 [01:49<02:18,  1.73it/s, Materializing param=model.layers.18.mlp.up_proj.weight]  
Loading weights:  46%|████▌     | 204/443 [01:49<02:32,  1.57it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:31,  1.58it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:18,  1.73it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:25,  1.64it/s, Materializing param=model.layers.18.mlp.up_proj.weight]  
Loading weights:  46%|████▌     | 204/443 [01:49<02:25,  1.64it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:53,  1.37it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
Loading weights:  46%|████▌     | 204/443 [01:49<02:53,  1.37it/s, Materializing param=model.layers.18.mlp.up_proj.weight]  
Loading weights:  46%|████▌     | 204/443 [01:49<02:53,  1.37it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:34,  1.11it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:24,  1.17it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:18,  1.20it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:35,  1.10it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:35,  1.10it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:34,  1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:18,  1.20it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:35,  1.10it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:34,  1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:18,  1.20it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:24,  1.17it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:24,  1.17it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<03:18,  1.20it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]        
Loading weights:  46%|████▋     | 205/443 [01:51<03:20,  1.19it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<03:18,  1.20it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:58,  1.00s/it, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:20,  1.19it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:58,  1.00s/it, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:20,  1.19it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:58,  1.00s/it, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<03:19,  1.19it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]        
Loading weights:  47%|████▋     | 206/443 [01:51<03:19,  1.19it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<03:17,  1.20it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<03:17,  1.20it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<03:57,  1.00s/it, Materializing param=model.layers.18.self_attn.k_norm.weight]        
Loading weights:  46%|████▋     | 205/443 [01:51<03:33,  1.11it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<03:57,  1.00s/it, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:34,  1.11it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:33,  1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<03:56,  1.00s/it, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:34,  1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<03:56,  1.00s/it, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:33,  1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  46%|████▋     | 205/443 [01:51<03:34,  1.11it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<03:33,  1.11it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]        
Loading weights:  47%|████▋     | 206/443 [01:51<03:33,  1.11it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]        
Loading weights:  47%|████▋     | 206/443 [01:51<03:33,  1.11it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<03:33,  1.11it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<03:32,  1.11it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<03:32,  1.11it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<03:32,  1.11it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<03:32,  1.11it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:13,  1.76it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<01:59,  1.96it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:05,  1.87it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:05,  1.87it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<02:50,  1.39it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:13,  1.76it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<02:56,  1.35it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<02:57,  1.34it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:13,  1.76it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:05,  1.87it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<01:59,  1.96it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:05,  1.87it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<02:56,  1.35it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]        
Loading weights:  47%|████▋     | 208/443 [01:51<02:05,  1.87it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<02:57,  1.34it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]        
Loading weights:  47%|████▋     | 206/443 [01:51<02:56,  1.35it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<01:59,  1.96it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<02:57,  1.34it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:05,  1.87it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<02:50,  1.39it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]        
Loading weights:  47%|████▋     | 207/443 [01:51<02:56,  1.34it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<02:56,  1.34it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<02:55,  1.35it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<02:55,  1.35it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:55,  1.34it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:55,  1.34it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 206/443 [01:51<02:50,  1.39it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:54,  1.35it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:54,  1.35it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<02:49,  1.39it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<02:49,  1.39it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:49,  1.39it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:49,  1.39it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<02:20,  1.68it/s, Materializing param=model.layers.18.self_attn.k_norm.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<02:20,  1.68it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 207/443 [01:51<02:20,  1.68it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:19,  1.68it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 208/443 [01:51<02:19,  1.68it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<02:05,  1.87it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<02:12,  1.76it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:56,  2.01it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<02:12,  1.76it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<02:04,  1.87it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:56,  2.01it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<02:04,  1.87it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:56,  2.01it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<02:05,  1.87it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:58,  1.97it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:55,  2.02it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:56,  2.00it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:58,  1.97it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:58,  1.97it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:49,  2.12it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:56,  2.00it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:59,  1.96it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:54,  2.03it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:56,  2.00it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:49,  2.12it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:54,  2.03it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:49,  2.12it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:49,  2.13it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:59,  1.96it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:58,  1.97it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:54,  2.03it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:58,  1.97it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:56,  2.00it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:49,  2.13it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:56,  2.00it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:49,  2.13it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:55,  2.02it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:46,  2.20it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:46,  2.20it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:46,  2.20it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 209/443 [01:52<01:55,  2.02it/s, Materializing param=model.layers.18.self_attn.q_norm.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:55,  2.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:55,  2.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:55,  2.02it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  47%|████▋     | 210/443 [01:52<01:55,  2.02it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:42,  2.27it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:59,  1.94it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:42,  2.26it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:42,  2.26it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:52,  2.07it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:59,  1.94it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:55,  2.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:42,  2.26it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:59,  1.94it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:42,  2.26it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:55,  2.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:42,  2.26it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:42,  2.26it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:52,  2.07it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:55,  2.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:55,  2.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:55,  2.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:52,  2.07it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:42,  2.27it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:55,  2.01it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:42,  2.27it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:43,  2.23it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:43,  2.23it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 211/443 [01:53<01:43,  2.23it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
Loading weights:  48%|████▊     | 212/443 [01:53<01:43,  2.23it/s, Materializing param=model.layers.19.input_layernorm.weight] 
Loading weights:  48%|████▊     | 212/443 [01:53<01:43,  2.23it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 213/443 [01:53<01:43,  2.23it/s, Materializing param=model.layers.19.mlp.down_proj.weight]  
Loading weights:  48%|████▊     | 213/443 [01:53<01:43,  2.23it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 212/443 [01:53<01:41,  2.27it/s, Materializing param=model.layers.19.input_layernorm.weight] 
Loading weights:  48%|████▊     | 212/443 [01:53<01:41,  2.27it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 212/443 [01:53<01:59,  1.94it/s, Materializing param=model.layers.19.input_layernorm.weight] 
Loading weights:  48%|████▊     | 212/443 [01:53<01:54,  2.01it/s, Materializing param=model.layers.19.input_layernorm.weight] 
Loading weights:  48%|████▊     | 212/443 [01:53<01:54,  2.01it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 212/443 [01:53<01:59,  1.94it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 213/443 [01:53<01:58,  1.94it/s, Materializing param=model.layers.19.mlp.down_proj.weight]  
Loading weights:  48%|████▊     | 213/443 [01:53<01:58,  1.94it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 212/443 [01:53<01:51,  2.07it/s, Materializing param=model.layers.19.input_layernorm.weight] 
Loading weights:  48%|████▊     | 212/443 [01:53<01:51,  2.07it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 212/443 [01:53<01:42,  2.26it/s, Materializing param=model.layers.19.input_layernorm.weight] 
Loading weights:  48%|████▊     | 212/443 [01:53<01:42,  2.26it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 212/443 [01:53<01:42,  2.26it/s, Materializing param=model.layers.19.input_layernorm.weight] 
Loading weights:  48%|████▊     | 212/443 [01:53<01:42,  2.26it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 213/443 [01:53<01:41,  2.26it/s, Materializing param=model.layers.19.mlp.down_proj.weight]  
Loading weights:  48%|████▊     | 213/443 [01:53<01:41,  2.26it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 212/443 [01:53<01:55,  2.01it/s, Materializing param=model.layers.19.input_layernorm.weight] 
Loading weights:  48%|████▊     | 212/443 [01:53<01:55,  2.01it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 213/443 [01:53<01:54,  2.01it/s, Materializing param=model.layers.19.mlp.down_proj.weight]  
Loading weights:  48%|████▊     | 213/443 [01:53<01:54,  2.01it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<01:59,  1.91it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<01:59,  1.91it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:09,  1.76it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<01:58,  1.93it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 213/443 [01:54<02:28,  1.55it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 213/443 [01:54<02:17,  1.67it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:07,  1.80it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<01:59,  1.91it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:09,  1.76it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<01:58,  1.93it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:07,  1.80it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 213/443 [01:54<02:28,  1.55it/s, Materializing param=model.layers.19.mlp.down_proj.weight]  
Loading weights:  48%|████▊     | 213/443 [01:54<02:17,  1.67it/s, Materializing param=model.layers.19.mlp.down_proj.weight]  
Loading weights:  48%|████▊     | 214/443 [01:54<01:58,  1.93it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:07,  1.80it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 213/443 [01:54<02:28,  1.55it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 213/443 [01:54<02:17,  1.67it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:09,  1.76it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:54<01:58,  1.93it/s, Materializing param=model.layers.19.mlp.up_proj.weight]  
Loading weights:  49%|████▊     | 215/443 [01:54<01:58,  1.93it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:28,  1.55it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:16,  1.67it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:16,  1.67it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:28,  1.55it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 213/443 [01:54<02:16,  1.68it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 213/443 [01:54<02:26,  1.57it/s, Materializing param=model.layers.19.input_layernorm.weight]
Loading weights:  48%|████▊     | 213/443 [01:54<02:26,  1.57it/s, Materializing param=model.layers.19.mlp.down_proj.weight]  
Loading weights:  48%|████▊     | 213/443 [01:54<02:26,  1.57it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 213/443 [01:54<02:16,  1.68it/s, Materializing param=model.layers.19.mlp.down_proj.weight]  
Loading weights:  48%|████▊     | 214/443 [01:54<02:25,  1.57it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:25,  1.57it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 213/443 [01:54<02:16,  1.68it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:15,  1.68it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  48%|████▊     | 214/443 [01:54<02:15,  1.68it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<02:04,  1.83it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<02:04,  1.83it/s, Materializing param=model.layers.19.mlp.up_proj.weight]  
Loading weights:  49%|████▊     | 215/443 [01:55<02:04,  1.83it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<01:58,  1.92it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<01:58,  1.92it/s, Materializing param=model.layers.19.mlp.up_proj.weight]  
Loading weights:  49%|████▊     | 215/443 [01:55<01:58,  1.92it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<01:56,  1.96it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<02:03,  1.85it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<02:03,  1.85it/s, Materializing param=model.layers.19.mlp.up_proj.weight]  
Loading weights:  49%|████▊     | 215/443 [01:55<02:03,  1.85it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<01:56,  1.96it/s, Materializing param=model.layers.19.mlp.up_proj.weight]  
Loading weights:  49%|████▊     | 215/443 [01:55<01:56,  1.96it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<01:57,  1.95it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<01:57,  1.95it/s, Materializing param=model.layers.19.mlp.up_proj.weight]  
Loading weights:  49%|████▊     | 215/443 [01:55<01:57,  1.95it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<02:02,  1.87it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<02:02,  1.87it/s, Materializing param=model.layers.19.mlp.up_proj.weight]  
Loading weights:  49%|████▊     | 215/443 [01:55<02:02,  1.87it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<02:12,  1.72it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
Loading weights:  49%|████▊     | 215/443 [01:55<02:12,  1.72it/s, Materializing param=model.layers.19.mlp.up_proj.weight]  
Loading weights:  49%|████▊     | 215/443 [01:55<02:12,  1.72it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:40,  1.41it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:40,  1.41it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:40,  1.41it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:38,  1.43it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:38,  1.43it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:28,  1.53it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:38,  1.43it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:39,  1.41it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]        
Loading weights:  49%|████▉     | 216/443 [01:57<02:37,  1.44it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:32,  1.49it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:47,  1.36it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:39,  1.41it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:28,  1.53it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:32,  1.49it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:37,  1.43it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]        
Loading weights:  49%|████▉     | 216/443 [01:57<02:47,  1.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:28,  1.53it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:32,  1.49it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:31,  1.50it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:37,  1.44it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:47,  1.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:47,  1.36it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:39,  1.41it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:37,  1.44it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:37,  1.43it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:31,  1.49it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]        
Loading weights:  49%|████▉     | 216/443 [01:57<02:47,  1.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:27,  1.53it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]        
Loading weights:  49%|████▉     | 217/443 [01:57<02:46,  1.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]        
Loading weights:  49%|████▉     | 217/443 [01:57<02:31,  1.49it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:47,  1.36it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:39,  1.41it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:46,  1.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:27,  1.53it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:31,  1.50it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:30,  1.49it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:36,  1.44it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]        
Loading weights:  49%|████▉     | 218/443 [01:57<02:30,  1.49it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:46,  1.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]        
Loading weights:  49%|████▉     | 218/443 [01:57<02:45,  1.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:36,  1.44it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:46,  1.36it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:45,  1.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:37,  1.43it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:37,  1.43it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:35,  1.44it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:45,  1.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 216/443 [01:57<02:31,  1.50it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:45,  1.36it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:35,  1.44it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 217/443 [01:57<02:30,  1.50it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]        
Loading weights:  49%|████▉     | 217/443 [01:57<02:30,  1.50it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:29,  1.50it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<02:29,  1.50it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:28,  2.52it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<01:49,  2.06it/s, Materializing param=model.layers.19.self_attn.k_norm.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<01:49,  2.06it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:28,  2.52it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 218/443 [01:57<01:49,  2.06it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:28,  2.52it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:32,  2.43it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:30,  2.47it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:34,  2.38it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:28,  2.52it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:31,  2.46it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:48,  2.06it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:33,  2.39it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:48,  2.06it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:28,  2.52it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:34,  2.38it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:31,  2.46it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:33,  2.39it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:30,  2.47it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:32,  2.43it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:28,  2.52it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:34,  2.38it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:31,  2.46it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:33,  2.39it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:30,  2.47it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  49%|████▉     | 219/443 [01:57<01:32,  2.43it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:37,  2.29it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:33,  2.39it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:33,  2.38it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:37,  2.29it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:33,  2.39it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:35,  2.33it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:37,  2.29it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:31,  2.43it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:33,  2.38it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:30,  2.47it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:24,  2.63it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:37,  2.29it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:24,  2.62it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:24,  2.63it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:31,  2.43it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:30,  2.47it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:37,  2.29it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:35,  2.33it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:24,  2.63it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:24,  2.62it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:24,  2.62it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:22,  2.69it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:22,  2.69it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:23,  2.66it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:22,  2.69it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:35,  2.33it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:23,  2.66it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:23,  2.66it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:35,  2.33it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:35,  2.33it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:36,  2.32it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:35,  2.33it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:36,  2.32it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:36,  2.32it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:35,  2.33it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 220/443 [01:57<01:35,  2.33it/s, Materializing param=model.layers.19.self_attn.q_norm.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:35,  2.32it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:35,  2.32it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:35,  2.33it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|████▉     | 221/443 [01:57<01:35,  2.33it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:34,  2.33it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:34,  2.33it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 223/443 [01:57<00:57,  3.79it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 223/443 [01:57<00:57,  3.79it/s, Materializing param=model.layers.20.input_layernorm.weight] 
Loading weights:  50%|█████     | 223/443 [01:57<00:57,  3.79it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  51%|█████     | 224/443 [01:57<00:57,  3.79it/s, Materializing param=model.layers.20.mlp.down_proj.weight]  
Loading weights:  51%|█████     | 224/443 [01:57<00:57,  3.79it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:09,  3.16it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:09,  3.16it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:09,  3.16it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:15,  2.94it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:15,  2.94it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:15,  2.94it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 223/443 [01:57<01:09,  3.16it/s, Materializing param=model.layers.20.input_layernorm.weight] 
Loading weights:  50%|█████     | 223/443 [01:57<01:09,  3.16it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  51%|█████     | 224/443 [01:57<01:09,  3.16it/s, Materializing param=model.layers.20.mlp.down_proj.weight]  
Loading weights:  51%|█████     | 224/443 [01:57<01:09,  3.16it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:15,  2.93it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:16,  2.90it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:15,  2.93it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:16,  2.90it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:15,  2.93it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:16,  2.90it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 223/443 [01:57<01:14,  2.94it/s, Materializing param=model.layers.20.input_layernorm.weight] 
Loading weights:  50%|█████     | 223/443 [01:57<01:14,  2.94it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  51%|█████     | 224/443 [01:57<01:14,  2.94it/s, Materializing param=model.layers.20.mlp.down_proj.weight]  
Loading weights:  51%|█████     | 224/443 [01:57<01:14,  2.94it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:13,  2.99it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:13,  2.99it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:13,  2.99it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:15,  2.93it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:15,  2.93it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:15,  2.93it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:10,  3.12it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:10,  3.12it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 222/443 [01:57<01:10,  3.12it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 223/443 [01:57<01:15,  2.90it/s, Materializing param=model.layers.20.input_layernorm.weight] 
Loading weights:  50%|█████     | 223/443 [01:57<01:15,  2.90it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  51%|█████     | 224/443 [01:57<01:15,  2.90it/s, Materializing param=model.layers.20.mlp.down_proj.weight]  
Loading weights:  51%|█████     | 224/443 [01:57<01:15,  2.90it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  50%|█████     | 223/443 [01:57<01:13,  2.99it/s, Materializing param=model.layers.20.input_layernorm.weight] 
Loading weights:  50%|█████     | 223/443 [01:57<01:15,  2.93it/s, Materializing param=model.layers.20.input_layernorm.weight] 
Loading weights:  50%|█████     | 223/443 [01:57<01:13,  2.99it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  50%|█████     | 223/443 [01:57<01:15,  2.93it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  50%|█████     | 223/443 [01:57<01:10,  3.12it/s, Materializing param=model.layers.20.input_layernorm.weight] 
Loading weights:  51%|█████     | 224/443 [01:57<01:13,  2.99it/s, Materializing param=model.layers.20.mlp.down_proj.weight]  
Loading weights:  51%|█████     | 224/443 [01:57<01:13,  2.99it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  50%|█████     | 223/443 [01:57<01:10,  3.12it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  51%|█████     | 224/443 [01:57<01:10,  3.12it/s, Materializing param=model.layers.20.mlp.down_proj.weight]  
Loading weights:  51%|█████     | 224/443 [01:57<01:10,  3.12it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:38,  2.21it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:40,  2.18it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:43,  2.10it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:44,  2.09it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:38,  2.21it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:43,  2.10it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:43,  2.10it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:44,  2.09it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:43,  2.10it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:38,  2.21it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:43,  2.10it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:40,  2.18it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:44,  2.09it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:38,  2.21it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:43,  2.10it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:40,  2.18it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 224/443 [01:59<02:00,  1.82it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  51%|█████     | 224/443 [01:59<02:00,  1.82it/s, Materializing param=model.layers.20.mlp.down_proj.weight]  
Loading weights:  51%|█████     | 225/443 [01:59<01:38,  2.21it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 224/443 [01:59<02:00,  1.82it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  51%|█████     | 226/443 [01:59<01:38,  2.21it/s, Materializing param=model.layers.20.mlp.up_proj.weight]  
Loading weights:  51%|█████     | 226/443 [01:59<01:38,  2.21it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:59,  1.82it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:59,  1.82it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<01:38,  2.21it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  50%|█████     | 223/443 [01:59<02:26,  1.51it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
Loading weights:  50%|█████     | 223/443 [01:59<02:26,  1.51it/s, Materializing param=model.layers.20.input_layernorm.weight] 
Loading weights:  50%|█████     | 223/443 [01:59<02:26,  1.51it/s, Materializing param=model.layers.20.input_layernorm.weight]
Loading weights:  51%|█████     | 224/443 [01:59<02:25,  1.51it/s, Materializing param=model.layers.20.mlp.down_proj.weight]  
Loading weights:  51%|█████     | 224/443 [01:59<02:25,  1.51it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<02:24,  1.51it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 225/443 [01:59<02:24,  1.51it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:35,  2.28it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:35,  2.28it/s, Materializing param=model.layers.20.mlp.up_proj.weight]  
Loading weights:  51%|█████     | 226/443 [02:00<01:35,  2.28it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:32,  2.36it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:32,  2.36it/s, Materializing param=model.layers.20.mlp.up_proj.weight]  
Loading weights:  51%|█████     | 226/443 [02:00<01:32,  2.36it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:51,  1.95it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:51,  1.95it/s, Materializing param=model.layers.20.mlp.up_proj.weight]  
Loading weights:  51%|█████     | 226/443 [02:00<01:51,  1.95it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:48,  2.00it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:48,  2.00it/s, Materializing param=model.layers.20.mlp.up_proj.weight]  
Loading weights:  51%|█████     | 226/443 [02:00<01:48,  2.00it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:41,  2.14it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:41,  2.14it/s, Materializing param=model.layers.20.mlp.up_proj.weight]  
Loading weights:  51%|█████     | 226/443 [02:00<01:41,  2.14it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:46,  2.04it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<01:46,  2.04it/s, Materializing param=model.layers.20.mlp.up_proj.weight]  
Loading weights:  51%|█████     | 226/443 [02:00<01:46,  2.04it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:41,  2.13it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:38,  2.18it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:45,  2.04it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:38,  2.18it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:45,  2.04it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:38,  2.18it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:45,  2.04it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████▏    | 228/443 [02:00<01:38,  2.18it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]        
Loading weights:  51%|█████▏    | 228/443 [02:00<01:45,  2.04it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]        
Loading weights:  51%|█████▏    | 228/443 [02:00<01:38,  2.18it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:41,  2.13it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████▏    | 228/443 [02:00<01:45,  2.04it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:41,  2.13it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  52%|█████▏    | 229/443 [02:00<01:44,  2.04it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 229/443 [02:00<01:44,  2.04it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  51%|█████▏    | 228/443 [02:00<01:41,  2.13it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]        
Loading weights:  51%|█████▏    | 228/443 [02:00<01:41,  2.13it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights:  52%|█████▏    | 229/443 [02:00<01:40,  2.13it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 229/443 [02:00<01:40,  2.13it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:35,  2.26it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:35,  2.26it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:35,  2.26it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████▏    | 228/443 [02:00<01:34,  2.26it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]        
Loading weights:  51%|█████▏    | 228/443 [02:00<01:34,  2.26it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights:  52%|█████▏    | 229/443 [02:00<01:34,  2.26it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 229/443 [02:00<01:34,  2.26it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:43,  2.10it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:43,  2.10it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:43,  2.10it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████▏    | 228/443 [02:00<01:42,  2.10it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]        
Loading weights:  51%|█████▏    | 228/443 [02:00<01:42,  2.10it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:37,  2.22it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:37,  2.22it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:46,  2.03it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:37,  2.22it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████     | 227/443 [02:00<01:46,  2.03it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████▏    | 228/443 [02:00<01:36,  2.22it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]        
Loading weights:  51%|█████     | 227/443 [02:00<01:46,  2.03it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████▏    | 228/443 [02:00<01:36,  2.22it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights:  51%|█████▏    | 228/443 [02:00<01:45,  2.03it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]        
Loading weights:  51%|█████▏    | 228/443 [02:00<01:45,  2.03it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights:  52%|█████▏    | 229/443 [02:00<01:45,  2.03it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 229/443 [02:00<01:45,  2.03it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<02:06,  1.71it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
Loading weights:  51%|█████     | 226/443 [02:00<02:06,  1.71it/s, Materializing param=model.layers.20.mlp.up_proj.weight]  
Loading weights:  51%|█████     | 226/443 [02:00<02:06,  1.71it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
Loading weights:  51%|█████     | 227/443 [02:00<02:06,  1.71it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████     | 227/443 [02:00<02:06,  1.71it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
Loading weights:  51%|█████▏    | 228/443 [02:00<02:05,  1.71it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]        
Loading weights:  51%|█████▏    | 228/443 [02:00<02:05,  1.71it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights:  52%|█████▏    | 229/443 [02:00<02:04,  1.71it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 229/443 [02:00<02:04,  1.71it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 229/443 [02:01<01:17,  2.78it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights:  52%|█████▏    | 229/443 [02:01<01:17,  2.78it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 229/443 [02:01<01:17,  2.78it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 229/443 [02:01<01:36,  2.22it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 229/443 [02:01<01:36,  2.22it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:07,  3.14it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<00:59,  3.60it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:06,  3.18it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:02,  3.42it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:05,  3.26it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<00:59,  3.60it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:02,  3.42it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:07,  3.14it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:06,  3.18it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<00:59,  3.60it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:06,  3.18it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:06,  3.18it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:07,  3.14it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:02,  3.42it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:06,  3.18it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:06,  3.18it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:05,  3.26it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:05,  3.26it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 229/443 [02:01<01:12,  2.94it/s, Materializing param=model.layers.20.self_attn.k_norm.weight]
Loading weights:  52%|█████▏    | 229/443 [02:01<01:12,  2.94it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 229/443 [02:01<01:12,  2.94it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:12,  2.94it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:12,  2.94it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:03,  3.33it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:06,  3.18it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:05,  3.24it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:04,  3.27it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:01,  3.47it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:06,  3.18it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<00:58,  3.62it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:01,  3.47it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:05,  3.24it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:04,  3.27it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:05,  3.24it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:01,  3.47it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<00:58,  3.62it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:04,  3.27it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<00:58,  3.62it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<00:57,  3.69it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<00:57,  3.69it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<00:57,  3.69it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<00:57,  3.69it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:03,  3.33it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<00:57,  3.69it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:03,  3.33it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<01:03,  3.33it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<01:03,  3.33it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:10,  3.04it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:10,  3.04it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 230/443 [02:01<01:10,  3.04it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:09,  3.04it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 231/443 [02:01<01:09,  3.04it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<01:09,  3.04it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<01:09,  3.04it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<00:53,  3.95it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<00:53,  3.95it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<00:53,  3.95it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<00:58,  3.61it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<00:58,  3.61it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:01<00:58,  3.61it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:08,  3.05it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:00,  3.49it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:04,  3.23it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:00,  3.49it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:02<01:17,  2.74it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:08,  3.05it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:00,  3.49it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:08,  3.05it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:04,  3.23it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:04,  3.23it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:02<01:21,  2.59it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 232/443 [02:02<01:17,  2.74it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:02<01:17,  2.74it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:02<01:21,  2.59it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:02<01:21,  2.59it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:20,  2.59it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:16,  2.74it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:20,  2.59it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:16,  2.74it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:02<01:14,  2.82it/s, Materializing param=model.layers.20.self_attn.q_norm.weight]
Loading weights:  52%|█████▏    | 232/443 [02:02<01:14,  2.82it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  52%|█████▏    | 232/443 [02:02<01:14,  2.82it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:14,  2.82it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:14,  2.82it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:05,  3.21it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:05,  3.21it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:01,  3.43it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:01,  3.43it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:05,  3.21it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 233/443 [02:02<01:01,  3.43it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
Loading weights:  53%|█████▎    | 234/443 [02:02<01:01,  3.43it/s, Materializing param=model.layers.21.input_layernorm.weight] 
Loading weights:  53%|█████▎    | 234/443 [02:02<01:01,  3.43it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|█████▎    | 234/443 [02:02<01:05,  3.21it/s, Materializing param=model.layers.21.input_layernorm.weight] 
Loading weights:  53%|█████▎    | 235/443 [02:02<01:00,  3.43it/s, Materializing param=model.layers.21.mlp.down_proj.weight]  
Loading weights:  53%|█████▎    | 235/443 [02:02<01:00,  3.43it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 234/443 [02:02<01:05,  3.21it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|█████▎    | 235/443 [02:02<01:04,  3.21it/s, Materializing param=model.layers.21.mlp.down_proj.weight]  
Loading weights:  53%|█████▎    | 235/443 [02:02<01:04,  3.21it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 234/443 [02:02<00:59,  3.49it/s, Materializing param=model.layers.21.input_layernorm.weight] 
Loading weights:  53%|█████▎    | 234/443 [02:02<01:08,  3.05it/s, Materializing param=model.layers.21.input_layernorm.weight] 
Loading weights:  53%|█████▎    | 234/443 [02:02<00:59,  3.49it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|█████▎    | 234/443 [02:02<01:08,  3.05it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|█████▎    | 235/443 [02:02<00:59,  3.49it/s, Materializing param=model.layers.21.mlp.down_proj.weight]  
Loading weights:  53%|█████▎    | 235/443 [02:02<01:08,  3.05it/s, Materializing param=model.layers.21.mlp.down_proj.weight]  
Loading weights:  53%|█████▎    | 235/443 [02:02<00:59,  3.49it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 235/443 [02:02<01:08,  3.05it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 234/443 [02:02<01:16,  2.74it/s, Materializing param=model.layers.21.input_layernorm.weight] 
Loading weights:  53%|█████▎    | 234/443 [02:02<01:16,  2.74it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|█████▎    | 235/443 [02:02<01:15,  2.74it/s, Materializing param=model.layers.21.mlp.down_proj.weight]  
Loading weights:  53%|█████▎    | 235/443 [02:02<01:15,  2.74it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 234/443 [02:02<01:14,  2.82it/s, Materializing param=model.layers.21.input_layernorm.weight] 
Loading weights:  53%|█████▎    | 234/443 [02:02<01:04,  3.23it/s, Materializing param=model.layers.21.input_layernorm.weight] 
Loading weights:  53%|█████▎    | 234/443 [02:02<01:20,  2.59it/s, Materializing param=model.layers.21.input_layernorm.weight] 
Loading weights:  53%|█████▎    | 234/443 [02:02<01:14,  2.82it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|█████▎    | 234/443 [02:02<01:04,  3.23it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|█████▎    | 234/443 [02:02<01:20,  2.59it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|█████▎    | 235/443 [02:02<01:04,  3.23it/s, Materializing param=model.layers.21.mlp.down_proj.weight]  
Loading weights:  53%|█████▎    | 235/443 [02:02<01:04,  3.23it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 235/443 [02:02<01:11,  2.92it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|█████▎    | 235/443 [02:02<01:11,  2.92it/s, Materializing param=model.layers.21.mlp.down_proj.weight]  
Loading weights:  53%|█████▎    | 235/443 [02:02<01:11,  2.92it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:26,  2.38it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:33,  2.22it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:33,  2.21it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:36,  2.16it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:29,  2.30it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:22,  2.50it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:26,  2.38it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:25,  2.42it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:33,  2.22it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:36,  2.16it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:26,  2.38it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:22,  2.50it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:33,  2.22it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 235/443 [02:03<01:32,  2.25it/s, Materializing param=model.layers.21.input_layernorm.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:36,  2.16it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:33,  2.21it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:22,  2.50it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:25,  2.42it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 235/443 [02:03<01:32,  2.25it/s, Materializing param=model.layers.21.mlp.down_proj.weight]  
Loading weights:  53%|█████▎    | 236/443 [02:03<01:33,  2.21it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:25,  2.42it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 235/443 [02:03<01:32,  2.25it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:03<01:32,  2.22it/s, Materializing param=model.layers.21.mlp.up_proj.weight]  
Loading weights:  53%|█████▎    | 237/443 [02:03<01:32,  2.22it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:29,  2.30it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:31,  2.25it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:31,  2.25it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 236/443 [02:03<01:29,  2.30it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:28,  2.33it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:28,  2.33it/s, Materializing param=model.layers.21.mlp.up_proj.weight]  
Loading weights:  53%|█████▎    | 237/443 [02:04<01:28,  2.33it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:26,  2.40it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:26,  2.40it/s, Materializing param=model.layers.21.mlp.up_proj.weight]  
Loading weights:  53%|█████▎    | 237/443 [02:04<01:26,  2.40it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:38,  2.09it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:38,  2.09it/s, Materializing param=model.layers.21.mlp.up_proj.weight]  
Loading weights:  53%|█████▎    | 237/443 [02:04<01:38,  2.09it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:27,  2.36it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:27,  2.36it/s, Materializing param=model.layers.21.mlp.up_proj.weight]  
Loading weights:  53%|█████▎    | 237/443 [02:04<01:27,  2.36it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:39,  2.07it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:35,  2.16it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:43,  1.98it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:35,  2.16it/s, Materializing param=model.layers.21.mlp.up_proj.weight]  
Loading weights:  53%|█████▎    | 237/443 [02:04<01:35,  2.16it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:43,  1.98it/s, Materializing param=model.layers.21.mlp.up_proj.weight]  
Loading weights:  53%|█████▎    | 237/443 [02:04<01:43,  1.98it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  53%|█████▎    | 237/443 [02:04<01:39,  2.07it/s, Materializing param=model.layers.21.mlp.up_proj.weight]  
Loading weights:  53%|█████▎    | 237/443 [02:04<01:39,  2.07it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:58,  1.72it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<02:01,  1.69it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:58,  1.72it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<02:01,  1.69it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<02:01,  1.69it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<02:05,  1.63it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:50,  1.85it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:52,  1.82it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:42,  2.00it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:56,  1.76it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:58,  1.72it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<02:05,  1.63it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:50,  1.85it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:56,  1.76it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:52,  1.82it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:42,  2.00it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:56,  1.76it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<02:00,  1.69it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]        
Loading weights:  54%|█████▎    | 238/443 [02:05<02:05,  1.63it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:50,  1.85it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:42,  2.00it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:52,  1.82it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<02:00,  1.69it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:56,  1.76it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<01:50,  1.85it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]        
Loading weights:  54%|█████▍    | 239/443 [02:05<01:58,  1.72it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]        
Loading weights:  54%|█████▍    | 239/443 [02:05<01:56,  1.76it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]        
Loading weights:  54%|█████▍    | 239/443 [02:05<01:50,  1.85it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<01:56,  1.76it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<01:58,  1.72it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<01:52,  1.82it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]        
Loading weights:  54%|█████▎    | 238/443 [02:05<01:56,  1.76it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<01:49,  1.85it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<01:52,  1.82it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<01:42,  2.00it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]        
Loading weights:  54%|█████▍    | 240/443 [02:05<01:55,  1.76it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<01:57,  1.72it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<01:42,  2.00it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<01:55,  1.76it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<01:49,  1.85it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<02:05,  1.63it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]        
Loading weights:  54%|█████▍    | 240/443 [02:05<02:00,  1.69it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<01:57,  1.72it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<02:05,  1.63it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<02:00,  1.69it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<01:51,  1.82it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▎    | 238/443 [02:05<01:56,  1.76it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<01:41,  2.00it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<01:51,  1.82it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<01:41,  2.00it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<02:04,  1.63it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<02:04,  1.63it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 239/443 [02:05<01:55,  1.76it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]        
Loading weights:  54%|█████▍    | 239/443 [02:05<01:55,  1.76it/s, Materializing param=model.layers.21.self_attn.k_norm.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<01:55,  1.76it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 240/443 [02:05<01:55,  1.76it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.06it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.02it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.06it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.04it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.02it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.06it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.02it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.04it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:05,  3.11it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.04it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:05,  3.11it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:05,  3.11it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:04,  3.15it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:08,  2.95it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:04,  3.15it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:04,  3.15it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:08,  2.95it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:08,  2.95it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:10,  2.86it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.04it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:10,  2.86it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:10,  2.86it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.04it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  54%|█████▍    | 241/443 [02:05<01:06,  3.04it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:12,  2.77it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:03,  3.15it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:13,  2.75it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:03,  3.15it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:13,  2.75it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:12,  2.79it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:13,  2.75it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:12,  2.75it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:00,  3.33it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:12,  2.79it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:12,  2.75it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:00,  3.33it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:12,  2.79it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:12,  2.75it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:00,  3.33it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:12,  2.75it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:12,  2.77it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:12,  2.75it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:11,  2.79it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:12,  2.75it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:11,  2.79it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:12,  2.77it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:12,  2.75it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:12,  2.77it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:12,  2.77it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:15,  2.65it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:14,  2.69it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:11,  2.81it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:15,  2.65it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:14,  2.69it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:11,  2.81it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:15,  2.65it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:14,  2.69it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 242/443 [02:06<01:11,  2.81it/s, Materializing param=model.layers.21.self_attn.q_norm.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:14,  2.69it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:15,  2.65it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:11,  2.81it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:15,  2.65it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:11,  2.81it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▍    | 243/443 [02:06<01:14,  2.69it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<01:15,  2.65it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<01:10,  2.81it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<01:15,  2.65it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<01:13,  2.69it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<01:10,  2.81it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<01:13,  2.69it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:53,  3.70it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:53,  3.70it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:53,  3.70it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:43,  4.52it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:46,  4.23it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:43,  4.52it/s, Materializing param=model.layers.22.input_layernorm.weight] 
Loading weights:  55%|█████▌    | 245/443 [02:06<00:46,  4.23it/s, Materializing param=model.layers.22.input_layernorm.weight] 
Loading weights:  55%|█████▌    | 245/443 [02:06<00:43,  4.52it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:46,  4.23it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:51,  3.85it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:44,  4.43it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:51,  3.85it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:44,  4.43it/s, Materializing param=model.layers.22.input_layernorm.weight] 
Loading weights:  55%|█████▌    | 244/443 [02:06<00:51,  3.85it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:44,  4.43it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:52,  3.82it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
Loading weights:  56%|█████▌    | 246/443 [02:06<00:44,  4.43it/s, Materializing param=model.layers.22.mlp.down_proj.weight]  
Loading weights:  56%|█████▌    | 246/443 [02:06<00:44,  4.43it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:52,  3.82it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 244/443 [02:06<00:52,  3.82it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.22.input_layernorm.weight] 
Loading weights:  55%|█████▌    | 245/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.22.input_layernorm.weight] 
Loading weights:  55%|█████▌    | 245/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  56%|█████▌    | 246/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.22.mlp.down_proj.weight]  
Loading weights:  56%|█████▌    | 246/443 [02:06<00:51,  3.83it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:53,  3.70it/s, Materializing param=model.layers.22.input_layernorm.weight] 
Loading weights:  55%|█████▌    | 245/443 [02:06<00:53,  3.70it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  56%|█████▌    | 246/443 [02:06<00:53,  3.70it/s, Materializing param=model.layers.22.mlp.down_proj.weight]  
Loading weights:  56%|█████▌    | 246/443 [02:06<00:53,  3.70it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:51,  3.82it/s, Materializing param=model.layers.22.input_layernorm.weight] 
Loading weights:  55%|█████▌    | 245/443 [02:06<00:51,  3.85it/s, Materializing param=model.layers.22.input_layernorm.weight] 
Loading weights:  55%|█████▌    | 245/443 [02:06<00:51,  3.85it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  55%|█████▌    | 245/443 [02:06<00:51,  3.82it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  56%|█████▌    | 246/443 [02:06<00:51,  3.82it/s, Materializing param=model.layers.22.mlp.down_proj.weight]  
Loading weights:  56%|█████▌    | 246/443 [02:06<00:51,  3.82it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:31,  2.14it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████▌    | 246/443 [02:08<01:46,  1.86it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:35,  2.04it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:30,  2.17it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:30,  2.16it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████▌    | 246/443 [02:08<00:46,  4.23it/s, Materializing param=model.layers.22.mlp.down_proj.weight]  
Loading weights:  56%|█████▌    | 247/443 [02:08<01:31,  2.14it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 246/443 [02:08<00:43,  4.52it/s, Materializing param=model.layers.22.mlp.down_proj.weight]  
Loading weights:  56%|█████▌    | 247/443 [02:08<01:30,  2.17it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 246/443 [02:08<01:46,  1.86it/s, Materializing param=model.layers.22.mlp.down_proj.weight]  
Loading weights:  56%|█████▌    | 246/443 [02:08<00:46,  4.23it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:35,  2.04it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:31,  2.14it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:30,  2.17it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:35,  2.04it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 246/443 [02:08<01:46,  1.86it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:35,  2.05it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:35,  2.05it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 246/443 [02:08<00:43,  4.52it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:30,  2.16it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:35,  2.05it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:45,  1.86it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:45,  1.86it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:34,  2.08it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:30,  2.16it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 246/443 [02:08<01:46,  1.84it/s, Materializing param=model.layers.22.input_layernorm.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:34,  2.08it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:34,  2.08it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 246/443 [02:08<01:46,  1.84it/s, Materializing param=model.layers.22.mlp.down_proj.weight]  
Loading weights:  56%|█████▌    | 246/443 [02:08<01:46,  1.84it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:46,  1.84it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 247/443 [02:08<01:46,  1.84it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:10,  1.49it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:17,  1.42it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:15,  1.44it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:23,  1.36it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:10,  1.49it/s, Materializing param=model.layers.22.mlp.up_proj.weight]  
Loading weights:  56%|█████▌    | 248/443 [02:10<02:15,  1.44it/s, Materializing param=model.layers.22.mlp.up_proj.weight]  
Loading weights:  56%|█████▌    | 248/443 [02:10<02:10,  1.49it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:23,  1.36it/s, Materializing param=model.layers.22.mlp.up_proj.weight]  
Loading weights:  56%|█████▌    | 248/443 [02:10<02:15,  1.44it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:21,  1.38it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:23,  1.36it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:17,  1.42it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:11,  1.48it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:21,  1.37it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:17,  1.42it/s, Materializing param=model.layers.22.mlp.up_proj.weight]  
Loading weights:  56%|█████▌    | 248/443 [02:10<02:21,  1.38it/s, Materializing param=model.layers.22.mlp.up_proj.weight]  
Loading weights:  56%|█████▌    | 248/443 [02:10<02:11,  1.48it/s, Materializing param=model.layers.22.mlp.up_proj.weight]  
Loading weights:  56%|█████▌    | 248/443 [02:10<02:17,  1.42it/s, Materializing param=model.layers.22.mlp.up_proj.weight]  
Loading weights:  56%|█████▌    | 248/443 [02:10<02:21,  1.37it/s, Materializing param=model.layers.22.mlp.up_proj.weight]  
Loading weights:  56%|█████▌    | 248/443 [02:10<02:21,  1.38it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:11,  1.48it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:17,  1.42it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:21,  1.37it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 248/443 [02:10<02:17,  1.42it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<03:00,  1.08it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<03:00,  1.08it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:54,  1.11it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:55,  1.10it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<03:00,  1.08it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:54,  1.11it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:50,  1.14it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:54,  1.11it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<03:02,  1.06it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:50,  1.14it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<02:59,  1.08it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]        
Loading weights:  56%|█████▌    | 249/443 [02:12<02:50,  1.14it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<02:59,  1.08it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:55,  1.10it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<03:02,  1.06it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<03:02,  1.06it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:55,  1.10it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:58,  1.08it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:55,  1.10it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<02:49,  1.14it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]        
Loading weights:  57%|█████▋    | 251/443 [02:12<02:58,  1.08it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<02:49,  1.14it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<03:01,  1.06it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]        
Loading weights:  56%|█████▋    | 250/443 [02:12<02:53,  1.11it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]        
Loading weights:  56%|█████▌    | 249/443 [02:12<02:49,  1.15it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<03:01,  1.06it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<02:53,  1.11it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<02:54,  1.10it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]        
Loading weights:  56%|█████▌    | 249/443 [02:12<02:55,  1.10it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<02:54,  1.10it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:49,  1.15it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<03:00,  1.06it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:52,  1.11it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:49,  1.15it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<03:00,  1.06it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:52,  1.11it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:53,  1.10it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:48,  1.14it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<03:00,  1.07it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:48,  1.14it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<02:48,  1.15it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]        
Loading weights:  57%|█████▋    | 251/443 [02:12<02:53,  1.10it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<02:48,  1.15it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<02:55,  1.10it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<03:00,  1.07it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  56%|█████▌    | 249/443 [02:12<03:00,  1.07it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:47,  1.15it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:47,  1.15it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<02:55,  1.10it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]        
Loading weights:  56%|█████▋    | 250/443 [02:12<02:59,  1.07it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]        
Loading weights:  56%|█████▋    | 250/443 [02:12<02:59,  1.07it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights:  56%|█████▋    | 250/443 [02:12<02:55,  1.10it/s, Materializing param=model.layers.22.self_attn.k_norm.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:58,  1.07it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:58,  1.07it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:54,  1.10it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 251/443 [02:12<02:54,  1.10it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<02:53,  1.10it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<03:00,  1.06it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<03:00,  1.06it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<02:47,  1.14it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:39,  1.92it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<02:47,  1.14it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:39,  1.92it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:39,  1.92it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<02:53,  1.10it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:39,  1.91it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:37,  1.95it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:38,  1.94it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:37,  1.95it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:39,  1.91it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:35,  1.99it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:37,  1.95it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:39,  1.91it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:35,  1.99it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:38,  1.94it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:35,  1.99it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 252/443 [02:12<01:38,  1.94it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:40,  1.89it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:37,  1.95it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:40,  1.89it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:38,  1.92it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:37,  1.95it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:40,  1.89it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:37,  1.95it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:38,  1.92it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:39,  1.92it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:37,  1.94it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:35,  1.99it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:36,  1.95it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:37,  1.95it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:39,  1.91it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:36,  1.95it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:37,  1.94it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:35,  1.99it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:37,  1.95it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:40,  1.89it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:39,  1.91it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:40,  1.89it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:22,  2.28it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:24,  2.24it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:24,  2.25it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:22,  2.28it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:24,  2.24it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:24,  2.25it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:22,  2.28it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:39,  1.92it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:24,  2.24it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:24,  2.25it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:25,  2.21it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:25,  2.21it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:25,  2.21it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 253/443 [02:12<01:39,  1.92it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:38,  1.92it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 254/443 [02:12<01:38,  1.92it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:25,  2.19it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:25,  2.19it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:25,  2.20it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:25,  2.19it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:23,  2.26it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:25,  2.20it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:23,  2.26it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  57%|█████▋    | 254/443 [02:13<01:39,  1.90it/s, Materializing param=model.layers.22.self_attn.q_norm.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:25,  2.20it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:25,  2.19it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:23,  2.26it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:26,  2.17it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:24,  2.23it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  57%|█████▋    | 254/443 [02:13<01:39,  1.90it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:25,  2.19it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  57%|█████▋    | 254/443 [02:13<01:39,  1.90it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:25,  2.19it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:39,  1.90it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:26,  2.17it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:24,  2.22it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:39,  1.90it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:26,  2.17it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:24,  2.22it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:24,  2.23it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:24,  2.22it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 255/443 [02:13<01:24,  2.23it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:14,  2.50it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.53it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:14,  2.51it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.53it/s, Materializing param=model.layers.23.input_layernorm.weight] 
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.53it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:14,  2.51it/s, Materializing param=model.layers.23.input_layernorm.weight] 
Loading weights:  58%|█████▊    | 256/443 [02:13<01:14,  2.51it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:14,  2.50it/s, Materializing param=model.layers.23.input_layernorm.weight] 
Loading weights:  58%|█████▊    | 257/443 [02:13<01:13,  2.51it/s, Materializing param=model.layers.23.mlp.down_proj.weight]  
Loading weights:  58%|█████▊    | 256/443 [02:13<01:14,  2.50it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.54it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 257/443 [02:13<01:13,  2.51it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.56it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.56it/s, Materializing param=model.layers.23.input_layernorm.weight] 
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.53it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.56it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:15,  2.48it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 257/443 [02:13<01:13,  2.53it/s, Materializing param=model.layers.23.mlp.down_proj.weight]  
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.54it/s, Materializing param=model.layers.23.input_layernorm.weight] 
Loading weights:  58%|█████▊    | 256/443 [02:13<01:15,  2.48it/s, Materializing param=model.layers.23.input_layernorm.weight] 
Loading weights:  58%|█████▊    | 257/443 [02:13<01:12,  2.56it/s, Materializing param=model.layers.23.mlp.down_proj.weight]  
Loading weights:  58%|█████▊    | 256/443 [02:13<01:15,  2.48it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  58%|█████▊    | 257/443 [02:13<01:12,  2.56it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:12,  2.56it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.54it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.53it/s, Materializing param=model.layers.23.input_layernorm.weight] 
Loading weights:  58%|█████▊    | 257/443 [02:13<01:13,  2.53it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 257/443 [02:13<01:15,  2.48it/s, Materializing param=model.layers.23.mlp.down_proj.weight]  
Loading weights:  58%|█████▊    | 257/443 [02:13<01:15,  2.48it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:13,  2.53it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  58%|█████▊    | 256/443 [02:13<01:12,  2.56it/s, Materializing param=model.layers.23.input_layernorm.weight] 
Loading weights:  58%|█████▊    | 257/443 [02:13<01:13,  2.54it/s, Materializing param=model.layers.23.mlp.down_proj.weight]  
Loading weights:  58%|█████▊    | 256/443 [02:13<01:12,  2.56it/s, Materializing param=model.layers.23.input_layernorm.weight]
Loading weights:  58%|█████▊    | 257/443 [02:13<01:13,  2.54it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 257/443 [02:13<01:14,  2.50it/s, Materializing param=model.layers.23.mlp.down_proj.weight]  
Loading weights:  58%|█████▊    | 257/443 [02:13<01:13,  2.53it/s, Materializing param=model.layers.23.mlp.down_proj.weight]  
Loading weights:  58%|█████▊    | 257/443 [02:13<01:12,  2.56it/s, Materializing param=model.layers.23.mlp.down_proj.weight]  
Loading weights:  58%|█████▊    | 257/443 [02:13<01:13,  2.53it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 257/443 [02:13<01:12,  2.56it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 257/443 [02:13<01:14,  2.50it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:52,  1.64it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:52,  1.64it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:52,  1.64it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:47,  1.72it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:47,  1.72it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:47,  1.72it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:48,  1.71it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:53,  1.64it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:49,  1.69it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:45,  1.75it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:52,  1.65it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:49,  1.69it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:53,  1.63it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:53,  1.64it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:45,  1.75it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:49,  1.69it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:52,  1.65it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:53,  1.64it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:45,  1.75it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:53,  1.63it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:52,  1.65it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:48,  1.71it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 258/443 [02:15<01:53,  1.63it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:15<01:44,  1.75it/s, Materializing param=model.layers.23.mlp.up_proj.weight]  
Loading weights:  58%|█████▊    | 258/443 [02:15<01:48,  1.71it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:15<01:44,  1.75it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:15<01:44,  1.76it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:15<01:44,  1.76it/s, Materializing param=model.layers.23.mlp.up_proj.weight]  
Loading weights:  58%|█████▊    | 259/443 [02:15<01:44,  1.76it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:15<01:48,  1.69it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:15<01:48,  1.69it/s, Materializing param=model.layers.23.mlp.up_proj.weight]  
Loading weights:  58%|█████▊    | 259/443 [02:15<01:48,  1.69it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:16<01:47,  1.72it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:16<01:47,  1.72it/s, Materializing param=model.layers.23.mlp.up_proj.weight]  
Loading weights:  58%|█████▊    | 259/443 [02:16<01:47,  1.72it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:16<02:02,  1.50it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:16<02:02,  1.50it/s, Materializing param=model.layers.23.mlp.up_proj.weight]  
Loading weights:  58%|█████▊    | 259/443 [02:16<02:02,  1.50it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:16<02:03,  1.49it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:16<02:03,  1.49it/s, Materializing param=model.layers.23.mlp.up_proj.weight]  
Loading weights:  58%|█████▊    | 259/443 [02:16<02:03,  1.49it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:16<01:57,  1.57it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:16<01:57,  1.57it/s, Materializing param=model.layers.23.mlp.up_proj.weight]  
Loading weights:  58%|█████▊    | 259/443 [02:16<01:57,  1.57it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:16<02:02,  1.50it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
Loading weights:  58%|█████▊    | 259/443 [02:16<02:02,  1.50it/s, Materializing param=model.layers.23.mlp.up_proj.weight]  
Loading weights:  58%|█████▊    | 259/443 [02:16<02:02,  1.50it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:47,  1.70it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:51,  1.65it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:54,  1.60it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:51,  1.65it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:51,  1.64it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:47,  1.70it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:51,  1.65it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:47,  1.70it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:54,  1.60it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:54,  1.60it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▉    | 261/443 [02:16<01:46,  1.70it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]        
Loading weights:  59%|█████▉    | 261/443 [02:16<01:46,  1.70it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights:  59%|█████▉    | 261/443 [02:16<01:53,  1.60it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]        
Loading weights:  59%|█████▊    | 260/443 [02:16<01:51,  1.64it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▉    | 261/443 [02:16<01:53,  1.60it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights:  59%|█████▉    | 261/443 [02:16<01:50,  1.65it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]        
Loading weights:  59%|█████▉    | 261/443 [02:16<01:50,  1.65it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:52,  1.60it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:51,  1.64it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:52,  1.60it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:46,  1.70it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:49,  1.65it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:46,  1.70it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:49,  1.65it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 261/443 [02:16<01:50,  1.64it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]        
Loading weights:  59%|█████▉    | 261/443 [02:16<01:50,  1.64it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:50,  1.64it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:50,  1.64it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:52,  1.63it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:52,  1.63it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:52,  1.63it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▉    | 261/443 [02:16<01:51,  1.63it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]        
Loading weights:  59%|█████▉    | 261/443 [02:16<01:51,  1.63it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:48,  1.69it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:50,  1.63it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:50,  1.63it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:48,  1.69it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:48,  1.69it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▉    | 261/443 [02:16<01:47,  1.69it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]        
Loading weights:  59%|█████▉    | 261/443 [02:16<01:47,  1.69it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:47,  1.69it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:47,  1.69it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:51,  1.64it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:52,  1.63it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:51,  1.64it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:52,  1.63it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:51,  1.64it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▊    | 260/443 [02:16<01:52,  1.63it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
Loading weights:  59%|█████▉    | 261/443 [02:16<01:51,  1.63it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]        
Loading weights:  59%|█████▉    | 261/443 [02:16<01:51,  1.64it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]        
Loading weights:  59%|█████▉    | 261/443 [02:16<01:51,  1.63it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights:  59%|█████▉    | 261/443 [02:16<01:51,  1.64it/s, Materializing param=model.layers.23.self_attn.k_norm.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:51,  1.63it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:50,  1.64it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:51,  1.63it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 262/443 [02:16<01:50,  1.64it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:46,  1.69it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:00,  2.96it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:07,  2.67it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:50,  1.63it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:49,  1.64it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:50,  1.63it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:49,  1.64it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:07,  2.67it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:00,  2.96it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:07,  2.67it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:00,  2.96it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:50,  1.63it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:46,  1.69it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:01,  2.91it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:50,  1.63it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:01,  2.91it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:01,  2.91it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:02,  2.86it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:02,  2.86it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  59%|█████▉    | 263/443 [02:16<01:02,  2.86it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:16<01:02,  2.86it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 264/443 [02:16<01:02,  2.86it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 265/443 [02:16<01:02,  2.86it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 265/443 [02:16<01:02,  2.86it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:16<00:53,  3.35it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:16<00:53,  3.35it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 264/443 [02:16<00:53,  3.35it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 265/443 [02:16<00:53,  3.35it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 265/443 [02:16<00:53,  3.35it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:54,  3.28it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:53,  3.35it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:53,  3.35it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:53,  3.35it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:54,  3.28it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:54,  3.28it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:53,  3.35it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:53,  3.35it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:54,  3.28it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:53,  3.33it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:54,  3.28it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:53,  3.33it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:53,  3.33it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:53,  3.33it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:53,  3.33it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:53,  3.32it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:53,  3.32it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:53,  3.32it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:53,  3.32it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:53,  3.32it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<01:07,  2.67it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<01:07,  2.67it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:52,  3.36it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:52,  3.36it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:52,  3.36it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:55,  3.20it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:55,  3.20it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 264/443 [02:17<00:55,  3.20it/s, Materializing param=model.layers.23.self_attn.q_norm.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:55,  3.20it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|█████▉    | 265/443 [02:17<00:55,  3.20it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:43,  4.03it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:52,  3.36it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:41,  4.27it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:44,  3.98it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:52,  3.36it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:42,  4.15it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:43,  4.03it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:41,  4.27it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:42,  4.14it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:43,  4.03it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:41,  4.31it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:41,  4.27it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:42,  4.15it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:42,  4.14it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:42,  4.15it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:42,  4.14it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:41,  4.31it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:41,  4.31it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:44,  3.98it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:44,  3.98it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:42,  4.13it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:42,  4.13it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 266/443 [02:17<00:42,  4.13it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 267/443 [02:17<00:41,  4.27it/s, Materializing param=model.layers.24.input_layernorm.weight] 
Loading weights:  60%|██████    | 267/443 [02:17<00:42,  4.14it/s, Materializing param=model.layers.24.input_layernorm.weight] 
Loading weights:  60%|██████    | 267/443 [02:17<00:42,  4.15it/s, Materializing param=model.layers.24.input_layernorm.weight] 
Loading weights:  60%|██████    | 267/443 [02:17<00:43,  4.03it/s, Materializing param=model.layers.24.input_layernorm.weight] 
Loading weights:  60%|██████    | 267/443 [02:17<00:41,  4.27it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  60%|██████    | 267/443 [02:17<00:44,  3.97it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
Loading weights:  60%|██████    | 267/443 [02:17<00:42,  4.14it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  60%|██████    | 267/443 [02:17<00:43,  4.03it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  60%|██████    | 267/443 [02:17<00:42,  4.15it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  60%|██████    | 267/443 [02:17<00:44,  3.97it/s, Materializing param=model.layers.24.input_layernorm.weight] 
Loading weights:  60%|██████    | 267/443 [02:17<00:44,  3.97it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  60%|██████    | 268/443 [02:17<00:42,  4.14it/s, Materializing param=model.layers.24.mlp.down_proj.weight]  
Loading weights:  60%|██████    | 268/443 [02:17<00:41,  4.27it/s, Materializing param=model.layers.24.mlp.down_proj.weight]  
Loading weights:  60%|██████    | 268/443 [02:17<00:42,  4.14it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  60%|██████    | 268/443 [02:17<00:41,  4.27it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  60%|██████    | 268/443 [02:17<00:43,  4.03it/s, Materializing param=model.layers.24.mlp.down_proj.weight]  
Loading weights:  60%|██████    | 268/443 [02:17<00:42,  4.15it/s, Materializing param=model.layers.24.mlp.down_proj.weight]  
Loading weights:  60%|██████    | 268/443 [02:17<00:44,  3.97it/s, Materializing param=model.layers.24.mlp.down_proj.weight]  
Loading weights:  60%|██████    | 268/443 [02:17<00:43,  4.03it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  60%|██████    | 267/443 [02:17<00:42,  4.13it/s, Materializing param=model.layers.24.input_layernorm.weight] 
Loading weights:  60%|██████    | 267/443 [02:17<00:40,  4.31it/s, Materializing param=model.layers.24.input_layernorm.weight] 
Loading weights:  60%|██████    | 268/443 [02:17<00:42,  4.15it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  60%|██████    | 267/443 [02:17<00:44,  3.98it/s, Materializing param=model.layers.24.input_layernorm.weight] 
Loading weights:  60%|██████    | 268/443 [02:17<00:44,  3.97it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  60%|██████    | 267/443 [02:17<00:42,  4.13it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  60%|██████    | 267/443 [02:17<00:40,  4.31it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  60%|██████    | 268/443 [02:17<00:42,  4.13it/s, Materializing param=model.layers.24.mlp.down_proj.weight]  
Loading weights:  60%|██████    | 268/443 [02:17<00:40,  4.31it/s, Materializing param=model.layers.24.mlp.down_proj.weight]  
Loading weights:  60%|██████    | 268/443 [02:17<00:42,  4.13it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  60%|██████    | 268/443 [02:17<00:40,  4.31it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  60%|██████    | 267/443 [02:17<00:44,  3.98it/s, Materializing param=model.layers.24.input_layernorm.weight]
Loading weights:  60%|██████    | 268/443 [02:17<00:43,  3.98it/s, Materializing param=model.layers.24.mlp.down_proj.weight]  
Loading weights:  60%|██████    | 268/443 [02:17<00:43,  3.98it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:15,  2.31it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:20,  2.15it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:15,  2.32it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.40it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:15,  2.31it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:15,  2.32it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.40it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:20,  2.15it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:15,  2.31it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:15,  2.32it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.40it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:20,  2.15it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 269/443 [02:19<01:12,  2.41it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:57,  1.48it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.63it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:52,  1.54it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:45,  1.65it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.62it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.63it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.62it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:52,  1.54it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.62it/s, Materializing param=model.layers.24.mlp.up_proj.weight]  
Loading weights:  61%|██████    | 270/443 [02:21<01:52,  1.54it/s, Materializing param=model.layers.24.mlp.up_proj.weight]  
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.63it/s, Materializing param=model.layers.24.mlp.up_proj.weight]  
Loading weights:  61%|██████    | 270/443 [02:21<01:57,  1.48it/s, Materializing param=model.layers.24.mlp.up_proj.weight]  
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.62it/s, Materializing param=model.layers.24.mlp.up_proj.weight]  
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.63it/s, Materializing param=model.layers.24.mlp.up_proj.weight]  
Loading weights:  61%|██████    | 270/443 [02:21<01:52,  1.54it/s, Materializing param=model.layers.24.mlp.up_proj.weight]  
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.62it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:52,  1.54it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.63it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:57,  1.48it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.62it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:46,  1.63it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:52,  1.54it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 270/443 [02:21<01:45,  1.65it/s, Materializing param=model.layers.24.mlp.up_proj.weight]  
Loading weights:  61%|██████    | 270/443 [02:21<01:45,  1.65it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:19,  1.24it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:23,  1.20it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:19,  1.24it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:19,  1.24it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:23,  1.20it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:10,  1.32it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:19,  1.23it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:23,  1.20it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]        
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:19,  1.23it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:22,  1.20it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]        
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:18,  1.24it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]        
Loading weights:  61%|██████    | 271/443 [02:22<02:19,  1.23it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:22,  1.20it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:18,  1.24it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]        
Loading weights:  61%|██████▏   | 272/443 [02:22<02:19,  1.23it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]        
Loading weights:  62%|██████▏   | 273/443 [02:22<02:21,  1.20it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:17,  1.24it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]        
Loading weights:  61%|██████▏   | 272/443 [02:22<02:19,  1.23it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:21,  1.20it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:12,  1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]        
Loading weights:  61%|██████▏   | 272/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:10,  1.32it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:17,  1.24it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:18,  1.23it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:18,  1.23it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  61%|██████    | 271/443 [02:22<02:10,  1.32it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:11,  1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:10,  1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:10,  1.30it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  61%|██████▏   | 272/443 [02:22<02:10,  1.32it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]        
Loading weights:  61%|██████▏   | 272/443 [02:22<02:10,  1.32it/s, Materializing param=model.layers.24.self_attn.k_norm.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:09,  1.32it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|██████▏   | 273/443 [02:22<02:09,  1.32it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:09,  1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:08,  1.32it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:20,  1.20it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:10,  1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:20,  1.20it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:10,  1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:08,  1.32it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:09,  1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:10,  1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:16,  1.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:10,  1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:16,  1.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:17,  1.23it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:10,  1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:10,  1.30it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 274/443 [02:22<02:17,  1.23it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:14,  2.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:14,  2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:16,  2.21it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:16,  2.21it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:14,  2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:16,  2.21it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:15,  2.24it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:15,  2.21it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:15,  2.21it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:14,  2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:14,  2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:14,  2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:14,  2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:14,  2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:14,  2.24it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:16,  2.19it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:16,  2.19it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:16,  2.19it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:16,  2.19it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:16,  2.19it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<01:15,  2.19it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<01:15,  2.19it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:18,  2.14it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:18,  2.14it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 275/443 [02:23<01:18,  2.14it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:18,  2.14it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:18,  2.14it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<01:17,  2.14it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<01:17,  2.14it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:06,  2.51it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:06,  2.51it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:57,  2.89it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:06,  2.51it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:57,  2.89it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<01:06,  2.51it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:57,  2.89it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<01:06,  2.51it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 278/443 [02:23<00:51,  3.21it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 278/443 [02:23<00:51,  3.21it/s, Materializing param=model.layers.25.input_layernorm.weight] 
Loading weights:  63%|██████▎   | 278/443 [02:23<00:51,  3.21it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  63%|██████▎   | 279/443 [02:23<00:51,  3.21it/s, Materializing param=model.layers.25.mlp.down_proj.weight]  
Loading weights:  63%|██████▎   | 279/443 [02:23<00:51,  3.21it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 278/443 [02:23<00:50,  3.27it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 278/443 [02:23<00:50,  3.27it/s, Materializing param=model.layers.25.input_layernorm.weight] 
Loading weights:  63%|██████▎   | 278/443 [02:23<00:50,  3.27it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  63%|██████▎   | 279/443 [02:23<00:50,  3.27it/s, Materializing param=model.layers.25.mlp.down_proj.weight]  
Loading weights:  63%|██████▎   | 279/443 [02:23<00:50,  3.27it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:57,  2.88it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:57,  2.88it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:57,  2.87it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:57,  2.88it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:57,  2.87it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:57,  2.87it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 278/443 [02:23<01:05,  2.51it/s, Materializing param=model.layers.25.input_layernorm.weight] 
Loading weights:  63%|██████▎   | 278/443 [02:23<01:05,  2.51it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:58,  2.85it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:58,  2.85it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<00:58,  2.85it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 278/443 [02:23<00:57,  2.89it/s, Materializing param=model.layers.25.input_layernorm.weight] 
Loading weights:  63%|██████▎   | 278/443 [02:23<00:57,  2.89it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  63%|██████▎   | 279/443 [02:23<00:56,  2.89it/s, Materializing param=model.layers.25.mlp.down_proj.weight]  
Loading weights:  63%|██████▎   | 279/443 [02:23<00:56,  2.89it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 278/443 [02:23<00:57,  2.87it/s, Materializing param=model.layers.25.input_layernorm.weight] 
Loading weights:  63%|██████▎   | 278/443 [02:23<00:57,  2.87it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  63%|██████▎   | 279/443 [02:23<00:57,  2.87it/s, Materializing param=model.layers.25.mlp.down_proj.weight]  
Loading weights:  63%|██████▎   | 279/443 [02:23<00:57,  2.87it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 278/443 [02:23<00:57,  2.88it/s, Materializing param=model.layers.25.input_layernorm.weight] 
Loading weights:  63%|██████▎   | 278/443 [02:23<00:57,  2.88it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  62%|██████▏   | 276/443 [02:23<01:07,  2.47it/s, Materializing param=model.layers.24.self_attn.q_norm.weight]
Loading weights:  63%|██████▎   | 279/443 [02:23<00:57,  2.88it/s, Materializing param=model.layers.25.mlp.down_proj.weight]  
Loading weights:  63%|██████▎   | 279/443 [02:23<00:57,  2.88it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 278/443 [02:23<00:57,  2.85it/s, Materializing param=model.layers.25.input_layernorm.weight] 
Loading weights:  62%|██████▏   | 276/443 [02:23<01:07,  2.47it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  63%|██████▎   | 278/443 [02:23<00:57,  2.85it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  63%|██████▎   | 279/443 [02:23<00:57,  2.85it/s, Materializing param=model.layers.25.mlp.down_proj.weight]  
Loading weights:  62%|██████▏   | 276/443 [02:23<01:07,  2.47it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
Loading weights:  63%|██████▎   | 279/443 [02:23<00:57,  2.85it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<01:07,  2.47it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 277/443 [02:23<01:07,  2.47it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
Loading weights:  63%|██████▎   | 278/443 [02:23<01:06,  2.47it/s, Materializing param=model.layers.25.input_layernorm.weight] 
Loading weights:  63%|██████▎   | 278/443 [02:23<01:06,  2.47it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  63%|██████▎   | 279/443 [02:23<01:06,  2.47it/s, Materializing param=model.layers.25.mlp.down_proj.weight]  
Loading weights:  63%|██████▎   | 279/443 [02:23<01:06,  2.47it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.19it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:11,  2.27it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.18it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:17,  2.09it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:18,  2.07it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.17it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.19it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 279/443 [02:25<01:22,  2.00it/s, Materializing param=model.layers.25.input_layernorm.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.19it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.19it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.18it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:17,  2.09it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 279/443 [02:25<01:22,  2.00it/s, Materializing param=model.layers.25.mlp.down_proj.weight]  
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.17it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.19it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.18it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.19it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:18,  2.07it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:17,  2.09it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 279/443 [02:25<01:22,  2.00it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:14,  2.17it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:18,  2.07it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:21,  2.00it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:11,  2.27it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:21,  2.00it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 280/443 [02:25<01:11,  2.27it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:46,  1.51it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:42,  1.59it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:46,  1.51it/s, Materializing param=model.layers.25.mlp.up_proj.weight]  
Loading weights:  63%|██████▎   | 281/443 [02:26<01:42,  1.59it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:46,  1.51it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:35,  1.70it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:38,  1.64it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:42,  1.59it/s, Materializing param=model.layers.25.mlp.up_proj.weight]  
Loading weights:  63%|██████▎   | 281/443 [02:26<01:46,  1.52it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:42,  1.59it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:42,  1.59it/s, Materializing param=model.layers.25.mlp.up_proj.weight]  
Loading weights:  63%|██████▎   | 281/443 [02:26<01:35,  1.70it/s, Materializing param=model.layers.25.mlp.up_proj.weight]  
Loading weights:  63%|██████▎   | 281/443 [02:26<01:46,  1.52it/s, Materializing param=model.layers.25.mlp.up_proj.weight]  
Loading weights:  63%|██████▎   | 281/443 [02:26<01:42,  1.59it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:35,  1.70it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:38,  1.64it/s, Materializing param=model.layers.25.mlp.up_proj.weight]  
Loading weights:  63%|██████▎   | 281/443 [02:26<01:38,  1.64it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:46,  1.52it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:42,  1.58it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:43,  1.57it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:43,  1.57it/s, Materializing param=model.layers.25.mlp.up_proj.weight]  
Loading weights:  63%|██████▎   | 281/443 [02:26<01:42,  1.58it/s, Materializing param=model.layers.25.mlp.up_proj.weight]  
Loading weights:  63%|██████▎   | 281/443 [02:26<01:43,  1.57it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:26<01:42,  1.57it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▎   | 282/443 [02:26<01:42,  1.57it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  63%|██████▎   | 281/443 [02:26<01:42,  1.58it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  64%|██████▍   | 283/443 [02:26<01:41,  1.57it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]        
Loading weights:  64%|██████▍   | 283/443 [02:26<01:41,  1.57it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights:  64%|██████▎   | 282/443 [02:26<01:41,  1.58it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▍   | 284/443 [02:26<01:41,  1.57it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 284/443 [02:26<01:41,  1.57it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:26<01:41,  1.58it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▍   | 283/443 [02:26<01:40,  1.58it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]        
Loading weights:  64%|██████▍   | 283/443 [02:26<01:40,  1.58it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights:  64%|██████▍   | 284/443 [02:26<01:40,  1.58it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 284/443 [02:26<01:40,  1.58it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:26<00:56,  2.80it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:26<00:56,  2.80it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:26<00:56,  2.80it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:26<00:56,  2.80it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:26<00:56,  2.80it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:26<00:56,  2.80it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:36,  1.66it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:36,  1.66it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:36,  1.66it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▍   | 283/443 [02:27<01:36,  1.66it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]        
Loading weights:  64%|██████▍   | 283/443 [02:27<01:36,  1.66it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:35,  1.66it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:35,  1.66it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:35,  1.66it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:35,  1.66it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:41,  1.59it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:41,  1.59it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:41,  1.59it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▍   | 283/443 [02:27<01:40,  1.59it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]        
Loading weights:  64%|██████▍   | 283/443 [02:27<01:40,  1.59it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:39,  1.59it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:39,  1.59it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:41,  1.58it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:41,  1.58it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:41,  1.58it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▍   | 283/443 [02:27<01:41,  1.58it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]        
Loading weights:  64%|██████▍   | 283/443 [02:27<01:41,  1.58it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:40,  1.58it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:40,  1.58it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:39,  1.58it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:39,  1.58it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:40,  1.60it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:40,  1.60it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:40,  1.60it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▍   | 283/443 [02:27<01:39,  1.60it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]        
Loading weights:  64%|██████▍   | 283/443 [02:27<01:39,  1.60it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:37,  1.65it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:37,  1.65it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:37,  1.65it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▍   | 283/443 [02:27<01:37,  1.65it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]        
Loading weights:  64%|██████▍   | 283/443 [02:27<01:37,  1.65it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:36,  1.65it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:36,  1.65it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:34,  1.70it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:34,  1.70it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▎   | 282/443 [02:27<01:34,  1.70it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
Loading weights:  64%|██████▍   | 283/443 [02:27<01:34,  1.70it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]        
Loading weights:  64%|██████▍   | 283/443 [02:27<01:34,  1.70it/s, Materializing param=model.layers.25.self_attn.k_norm.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:33,  1.70it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:33,  1.70it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:35,  1.65it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:39,  1.60it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 284/443 [02:27<01:39,  1.60it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:35,  1.65it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:38,  1.60it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:38,  1.60it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:32,  1.70it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:32,  1.70it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<00:52,  3.02it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<01:35,  1.65it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<00:52,  3.02it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<00:52,  3.02it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<01:35,  1.65it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<00:56,  2.80it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<01:37,  1.60it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<00:55,  2.80it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<00:51,  3.06it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<01:37,  1.60it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<00:56,  2.80it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:51,  3.02it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<00:55,  2.80it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<00:51,  3.06it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:51,  3.02it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<00:51,  3.06it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:52,  2.98it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<01:37,  1.60it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:52,  2.98it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:52,  2.98it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<01:37,  1.60it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:52,  2.98it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:52,  2.98it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:52,  2.98it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:50,  3.06it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:50,  3.06it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:00,  2.61it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:00,  2.61it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  64%|██████▍   | 285/443 [02:27<01:00,  2.61it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<01:00,  2.61it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 286/443 [02:27<01:00,  2.61it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:59,  2.61it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:27<00:59,  2.61it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:28<00:54,  2.88it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:52,  2.98it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:49,  3.15it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:47,  3.27it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:48,  3.17it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:49,  3.12it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:52,  2.98it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:52,  2.98it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:48,  3.17it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:49,  3.15it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:47,  3.27it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:49,  3.12it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:52,  2.98it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:48,  3.17it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:49,  3.15it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:47,  3.27it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:49,  3.12it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:28<00:54,  2.88it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:28<00:54,  2.88it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:53,  2.88it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<00:53,  2.88it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▍   | 286/443 [02:28<01:00,  2.58it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
Loading weights:  65%|██████▍   | 286/443 [02:28<01:00,  2.58it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 286/443 [02:28<01:00,  2.58it/s, Materializing param=model.layers.25.self_attn.q_norm.weight]
Loading weights:  65%|██████▍   | 287/443 [02:28<01:00,  2.58it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▍   | 287/443 [02:28<01:00,  2.58it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<01:00,  2.58it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 288/443 [02:28<01:00,  2.58it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:42,  3.62it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:42,  3.60it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:59,  2.58it/s, Materializing param=model.layers.26.input_layernorm.weight] 
Loading weights:  65%|██████▌   | 289/443 [02:28<00:42,  3.62it/s, Materializing param=model.layers.26.input_layernorm.weight] 
Loading weights:  65%|██████▌   | 289/443 [02:28<00:44,  3.47it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:59,  2.58it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:52,  2.95it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:44,  3.44it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:42,  3.62it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:44,  3.47it/s, Materializing param=model.layers.26.input_layernorm.weight] 
Loading weights:  65%|██████▌   | 289/443 [02:28<00:44,  3.44it/s, Materializing param=model.layers.26.input_layernorm.weight] 
Loading weights:  65%|██████▌   | 289/443 [02:28<00:52,  2.95it/s, Materializing param=model.layers.26.input_layernorm.weight] 
Loading weights:  65%|██████▌   | 289/443 [02:28<00:44,  3.44it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:52,  2.95it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:44,  3.47it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:42,  3.60it/s, Materializing param=model.layers.26.input_layernorm.weight] 
Loading weights:  65%|██████▌   | 290/443 [02:28<00:51,  2.95it/s, Materializing param=model.layers.26.mlp.down_proj.weight]  
Loading weights:  65%|██████▌   | 290/443 [02:28<00:51,  2.95it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  65%|██████▌   | 290/443 [02:28<00:44,  3.47it/s, Materializing param=model.layers.26.mlp.down_proj.weight]  
Loading weights:  65%|██████▌   | 289/443 [02:28<00:44,  3.50it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 290/443 [02:28<00:44,  3.47it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:44,  3.50it/s, Materializing param=model.layers.26.input_layernorm.weight] 
Loading weights:  65%|██████▌   | 290/443 [02:28<00:59,  2.58it/s, Materializing param=model.layers.26.mlp.down_proj.weight]  
Loading weights:  65%|██████▌   | 289/443 [02:28<00:44,  3.50it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:52,  2.95it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:42,  3.60it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  65%|██████▌   | 290/443 [02:28<00:43,  3.50it/s, Materializing param=model.layers.26.mlp.down_proj.weight]  
Loading weights:  65%|██████▌   | 290/443 [02:28<00:59,  2.58it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  65%|██████▌   | 290/443 [02:28<00:43,  3.50it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  65%|██████▌   | 289/443 [02:28<00:52,  2.95it/s, Materializing param=model.layers.26.input_layernorm.weight] 
Loading weights:  65%|██████▌   | 289/443 [02:28<00:52,  2.95it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  65%|██████▌   | 290/443 [02:28<00:51,  2.95it/s, Materializing param=model.layers.26.mlp.down_proj.weight]  
Loading weights:  65%|██████▌   | 290/443 [02:28<00:51,  2.95it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  65%|██████▌   | 290/443 [02:28<00:42,  3.60it/s, Materializing param=model.layers.26.mlp.down_proj.weight]  
Loading weights:  65%|██████▌   | 290/443 [02:28<00:42,  3.60it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:10,  2.15it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:19,  1.91it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:17,  1.95it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:18,  1.94it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:19,  1.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:01,  2.48it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:19,  1.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:19,  1.91it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:18,  1.94it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  65%|██████▌   | 290/443 [02:30<01:30,  1.68it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:17,  1.95it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:01,  2.48it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  65%|██████▌   | 290/443 [02:30<01:24,  1.80it/s, Materializing param=model.layers.26.input_layernorm.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:19,  1.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:18,  1.94it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:17,  1.95it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:01,  2.48it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  65%|██████▌   | 290/443 [02:30<01:30,  1.68it/s, Materializing param=model.layers.26.mlp.down_proj.weight]  
Loading weights:  66%|██████▌   | 291/443 [02:30<01:19,  1.91it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:10,  2.15it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  65%|██████▌   | 290/443 [02:30<01:30,  1.68it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  65%|██████▌   | 290/443 [02:30<01:24,  1.80it/s, Materializing param=model.layers.26.mlp.down_proj.weight]  
Loading weights:  66%|██████▌   | 291/443 [02:30<01:10,  2.15it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  65%|██████▌   | 290/443 [02:30<01:24,  1.80it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:24,  1.80it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:30,  1.68it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:24,  1.80it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 291/443 [02:30<01:30,  1.68it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:10,  2.15it/s, Materializing param=model.layers.26.mlp.up_proj.weight]  
Loading weights:  66%|██████▌   | 292/443 [02:30<01:10,  2.15it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:02,  2.42it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:02,  2.42it/s, Materializing param=model.layers.26.mlp.up_proj.weight]  
Loading weights:  66%|██████▌   | 292/443 [02:30<01:02,  2.42it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:16,  1.97it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:16,  1.97it/s, Materializing param=model.layers.26.mlp.up_proj.weight]  
Loading weights:  66%|██████▌   | 292/443 [02:30<01:16,  1.97it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:16,  1.98it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:16,  1.98it/s, Materializing param=model.layers.26.mlp.up_proj.weight]  
Loading weights:  66%|██████▌   | 292/443 [02:30<01:16,  1.98it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:23,  1.81it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:23,  1.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight]  
Loading weights:  66%|██████▌   | 292/443 [02:30<01:23,  1.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:14,  2.03it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:14,  2.03it/s, Materializing param=model.layers.26.mlp.up_proj.weight]  
Loading weights:  66%|██████▌   | 292/443 [02:30<01:14,  2.03it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:23,  1.81it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:30<01:23,  1.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight]  
Loading weights:  66%|██████▌   | 292/443 [02:30<01:23,  1.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:14,  2.01it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:21,  1.83it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:21,  1.83it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:07,  2.21it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:21,  1.83it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:07,  2.21it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▋   | 294/443 [02:31<01:21,  1.83it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]        
Loading weights:  66%|██████▌   | 293/443 [02:31<01:07,  2.21it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:14,  2.01it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▋   | 294/443 [02:31<01:21,  1.83it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights:  66%|██████▋   | 294/443 [02:31<01:07,  2.21it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]        
Loading weights:  67%|██████▋   | 295/443 [02:31<01:20,  1.83it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:20,  1.83it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  66%|██████▋   | 294/443 [02:31<01:07,  2.21it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:14,  2.01it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:06,  2.21it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:06,  2.21it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  66%|██████▋   | 294/443 [02:31<01:14,  2.01it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]        
Loading weights:  66%|██████▋   | 294/443 [02:31<01:14,  2.01it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:13,  2.01it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:13,  2.01it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:16,  1.97it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:16,  1.97it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:16,  1.97it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▋   | 294/443 [02:31<01:15,  1.97it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]        
Loading weights:  66%|██████▋   | 294/443 [02:31<01:15,  1.97it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:14,  1.97it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:14,  1.97it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:20,  1.86it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:20,  1.86it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:20,  1.87it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:20,  1.86it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:20,  1.87it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:20,  1.87it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▋   | 294/443 [02:31<01:19,  1.87it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]        
Loading weights:  66%|██████▋   | 294/443 [02:31<01:19,  1.87it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights:  66%|██████▋   | 294/443 [02:31<01:20,  1.86it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]        
Loading weights:  66%|██████▋   | 294/443 [02:31<01:20,  1.86it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:19,  1.87it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:19,  1.87it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:19,  1.86it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:19,  1.86it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:13,  2.03it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:13,  2.03it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:13,  2.03it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▋   | 294/443 [02:31<01:13,  2.03it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]        
Loading weights:  66%|██████▋   | 294/443 [02:31<01:13,  2.03it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:12,  2.03it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:12,  2.03it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<01:18,  1.87it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<01:19,  1.86it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<01:14,  1.97it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<00:44,  3.27it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<00:48,  3.03it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<00:43,  3.41it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<01:12,  2.03it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<01:18,  1.87it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<01:14,  1.97it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<01:19,  1.86it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<01:12,  2.03it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<00:44,  3.27it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<00:44,  3.27it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<00:43,  3.41it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<00:43,  3.41it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<00:48,  3.03it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<00:48,  3.03it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:31<01:39,  1.52it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
Loading weights:  66%|██████▌   | 292/443 [02:31<01:39,  1.52it/s, Materializing param=model.layers.26.mlp.up_proj.weight]  
Loading weights:  66%|██████▌   | 292/443 [02:31<01:39,  1.52it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:38,  1.52it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▌   | 293/443 [02:31<01:38,  1.52it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
Loading weights:  66%|██████▋   | 294/443 [02:31<01:37,  1.52it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]        
Loading weights:  66%|██████▋   | 294/443 [02:31<01:37,  1.52it/s, Materializing param=model.layers.26.self_attn.k_norm.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:37,  1.52it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 295/443 [02:31<01:37,  1.52it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<01:36,  1.52it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 296/443 [02:31<01:36,  1.52it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:44,  3.27it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:47,  3.06it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:44,  3.27it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:47,  3.06it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:47,  3.06it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:47,  3.06it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:48,  3.03it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:42,  3.41it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:52,  2.80it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:42,  3.41it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:47,  3.06it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:48,  2.99it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:47,  3.06it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:48,  2.99it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:52,  2.80it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:42,  3.44it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:52,  2.80it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:47,  3.06it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:48,  2.99it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:47,  3.06it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:42,  3.44it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:48,  3.03it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:42,  3.44it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:51,  2.80it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:48,  2.99it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:51,  2.80it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:48,  2.99it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:46,  3.15it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:43,  3.35it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:43,  3.35it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:43,  3.35it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:46,  3.15it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:46,  3.15it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:50,  2.90it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:50,  2.90it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 297/443 [02:32<00:50,  2.90it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:50,  2.90it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:50,  2.90it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:49,  2.90it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:49,  2.90it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  68%|██████▊   | 300/443 [02:32<00:49,  2.90it/s, Materializing param=model.layers.27.input_layernorm.weight] 
Loading weights:  68%|██████▊   | 300/443 [02:32<00:49,  2.90it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|██████▊   | 301/443 [02:32<00:49,  2.90it/s, Materializing param=model.layers.27.mlp.down_proj.weight]  
Loading weights:  68%|██████▊   | 301/443 [02:32<00:49,  2.90it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:40,  3.59it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:40,  3.59it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:40,  3.59it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:37,  3.80it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:37,  3.80it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:37,  3.80it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:39,  3.65it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:39,  3.65it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:39,  3.65it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:42,  3.38it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:42,  3.38it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:42,  3.38it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:38,  3.73it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:38,  3.73it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:42,  3.36it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:38,  3.73it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:42,  3.36it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:42,  3.36it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:44,  3.27it/s, Materializing param=model.layers.26.self_attn.q_norm.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:44,  3.27it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 298/443 [02:32<00:44,  3.27it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:44,  3.27it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  67%|██████▋   | 299/443 [02:32<00:44,  3.27it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  68%|██████▊   | 300/443 [02:32<00:38,  3.73it/s, Materializing param=model.layers.27.input_layernorm.weight] 
Loading weights:  68%|██████▊   | 300/443 [02:32<00:43,  3.27it/s, Materializing param=model.layers.27.input_layernorm.weight] 
Loading weights:  68%|██████▊   | 300/443 [02:32<00:43,  3.27it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|██████▊   | 300/443 [02:32<00:38,  3.73it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|██████▊   | 301/443 [02:32<00:43,  3.27it/s, Materializing param=model.layers.27.mlp.down_proj.weight]  
Loading weights:  68%|██████▊   | 301/443 [02:32<00:43,  3.27it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 301/443 [02:32<00:38,  3.73it/s, Materializing param=model.layers.27.mlp.down_proj.weight]  
Loading weights:  68%|██████▊   | 301/443 [02:32<00:38,  3.73it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 300/443 [02:32<00:37,  3.80it/s, Materializing param=model.layers.27.input_layernorm.weight] 
Loading weights:  68%|██████▊   | 300/443 [02:32<00:37,  3.80it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|██████▊   | 300/443 [02:32<00:39,  3.65it/s, Materializing param=model.layers.27.input_layernorm.weight] 
Loading weights:  68%|██████▊   | 300/443 [02:32<00:39,  3.59it/s, Materializing param=model.layers.27.input_layernorm.weight] 
Loading weights:  68%|██████▊   | 300/443 [02:32<00:39,  3.59it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|██████▊   | 300/443 [02:32<00:39,  3.65it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|██████▊   | 301/443 [02:32<00:38,  3.65it/s, Materializing param=model.layers.27.mlp.down_proj.weight]  
Loading weights:  68%|██████▊   | 301/443 [02:32<00:38,  3.65it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 301/443 [02:32<00:39,  3.59it/s, Materializing param=model.layers.27.mlp.down_proj.weight]  
Loading weights:  68%|██████▊   | 301/443 [02:32<00:39,  3.59it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 300/443 [02:32<00:42,  3.36it/s, Materializing param=model.layers.27.input_layernorm.weight] 
Loading weights:  68%|██████▊   | 300/443 [02:32<00:42,  3.36it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|██████▊   | 301/443 [02:33<00:52,  2.69it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|██████▊   | 301/443 [02:33<00:52,  2.69it/s, Materializing param=model.layers.27.mlp.down_proj.weight]  
Loading weights:  68%|██████▊   | 302/443 [02:33<00:44,  3.19it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:46,  3.04it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:47,  2.96it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 301/443 [02:33<00:52,  2.69it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:44,  3.19it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:46,  3.04it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:47,  2.96it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 301/443 [02:33<00:57,  2.47it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:44,  3.19it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:47,  2.96it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:46,  3.04it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:52,  2.69it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:52,  2.69it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 301/443 [02:33<00:57,  2.47it/s, Materializing param=model.layers.27.mlp.down_proj.weight]  
Loading weights:  68%|██████▊   | 301/443 [02:33<00:57,  2.47it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:57,  2.47it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:57,  2.47it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:49,  2.85it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:49,  2.85it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:49,  2.85it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:33<00:49,  2.85it/s, Materializing param=model.layers.27.mlp.up_proj.weight]  
Loading weights:  68%|██████▊   | 303/443 [02:33<00:49,  2.85it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  68%|██████▊   | 300/443 [02:33<01:03,  2.25it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
Loading weights:  68%|██████▊   | 300/443 [02:33<01:03,  2.25it/s, Materializing param=model.layers.27.input_layernorm.weight] 
Loading weights:  68%|██████▊   | 300/443 [02:33<01:03,  2.25it/s, Materializing param=model.layers.27.input_layernorm.weight]
Loading weights:  68%|██████▊   | 301/443 [02:33<01:03,  2.25it/s, Materializing param=model.layers.27.mlp.down_proj.weight]  
Loading weights:  68%|██████▊   | 301/443 [02:33<01:03,  2.25it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<01:02,  2.25it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<01:02,  2.25it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:47,  3.00it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:47,  3.00it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 302/443 [02:33<00:47,  3.00it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:33<00:46,  3.00it/s, Materializing param=model.layers.27.mlp.up_proj.weight]  
Loading weights:  68%|██████▊   | 303/443 [02:33<00:46,  3.00it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:50,  2.79it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:50,  2.79it/s, Materializing param=model.layers.27.mlp.up_proj.weight]  
Loading weights:  68%|██████▊   | 303/443 [02:34<00:50,  2.79it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:50,  2.76it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:50,  2.76it/s, Materializing param=model.layers.27.mlp.up_proj.weight]  
Loading weights:  68%|██████▊   | 303/443 [02:34<00:50,  2.76it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:48,  2.91it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:48,  2.91it/s, Materializing param=model.layers.27.mlp.up_proj.weight]  
Loading weights:  68%|██████▊   | 303/443 [02:34<00:48,  2.91it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:52,  2.69it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:52,  2.69it/s, Materializing param=model.layers.27.mlp.up_proj.weight]  
Loading weights:  68%|██████▊   | 303/443 [02:34<00:52,  2.69it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:49,  2.84it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:49,  2.84it/s, Materializing param=model.layers.27.mlp.up_proj.weight]  
Loading weights:  68%|██████▊   | 303/443 [02:34<00:49,  2.84it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:48,  2.90it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
Loading weights:  68%|██████▊   | 303/443 [02:34<00:48,  2.90it/s, Materializing param=model.layers.27.mlp.up_proj.weight]  
Loading weights:  68%|██████▊   | 303/443 [02:34<00:48,  2.90it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:10,  1.97it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:12,  1.91it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:13,  1.90it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:09,  2.00it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:01,  2.26it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:09,  2.00it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:12,  1.91it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:12,  1.91it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:14,  1.86it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:10,  1.97it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:06,  2.09it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:01,  2.26it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:10,  1.97it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:09,  2.01it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:09,  2.00it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▉   | 305/443 [02:35<01:12,  1.91it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]        
Loading weights:  69%|██████▊   | 304/443 [02:35<01:06,  2.09it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:09,  2.01it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:06,  2.09it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:01,  2.26it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:09,  2.01it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▉   | 305/443 [02:35<01:12,  1.91it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights:  69%|██████▉   | 305/443 [02:35<01:10,  1.97it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]        
Loading weights:  69%|██████▊   | 304/443 [02:35<01:14,  1.86it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▉   | 305/443 [02:35<01:10,  1.97it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:14,  1.86it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▉   | 305/443 [02:35<01:06,  2.09it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]        
Loading weights:  69%|██████▉   | 305/443 [02:35<01:08,  2.00it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]        
Loading weights:  69%|██████▊   | 304/443 [02:35<01:13,  1.90it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▉   | 305/443 [02:35<01:01,  2.26it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]        
Loading weights:  69%|██████▉   | 305/443 [02:35<01:06,  2.09it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights:  69%|██████▉   | 305/443 [02:35<01:08,  2.00it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:09,  1.97it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 305/443 [02:35<01:08,  2.01it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]        
Loading weights:  69%|██████▉   | 305/443 [02:35<01:01,  2.26it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights:  69%|██████▉   | 305/443 [02:35<01:14,  1.86it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]        
Loading weights:  69%|██████▉   | 305/443 [02:35<01:08,  2.01it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:05,  2.09it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:09,  1.97it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▊   | 304/443 [02:35<01:13,  1.90it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
Loading weights:  69%|██████▉   | 305/443 [02:35<01:14,  1.86it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:05,  2.09it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:08,  2.00it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:00,  2.26it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:11,  1.91it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:08,  2.00it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:00,  2.26it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:11,  1.91it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:13,  1.86it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:08,  2.01it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:13,  1.86it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:08,  2.01it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 305/443 [02:35<01:12,  1.90it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]        
Loading weights:  69%|██████▉   | 305/443 [02:35<01:12,  1.90it/s, Materializing param=model.layers.27.self_attn.k_norm.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:11,  1.90it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 306/443 [02:35<01:11,  1.90it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.12it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:45,  3.00it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:42,  3.21it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.12it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:42,  3.22it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:45,  3.00it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.12it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.12it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.11it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:45,  3.00it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.12it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:42,  3.22it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.12it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:42,  3.22it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.10it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.10it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.10it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.11it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:42,  3.21it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:42,  3.21it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:43,  3.11it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:42,  3.19it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:42,  3.19it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  69%|██████▉   | 307/443 [02:35<00:42,  3.19it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:42,  3.19it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:42,  3.19it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:41,  3.19it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:41,  3.19it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:40,  3.37it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:40,  3.37it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:39,  3.45it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:40,  3.37it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:39,  3.45it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:39,  3.45it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:39,  3.37it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:39,  3.37it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:38,  3.45it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:38,  3.45it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:40,  3.36it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:41,  3.23it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:41,  3.23it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:41,  3.23it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:40,  3.36it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:41,  3.23it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:41,  3.23it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:40,  3.36it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:39,  3.36it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:39,  3.36it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:39,  3.44it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:39,  3.44it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:39,  3.44it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:40,  3.30it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:40,  3.30it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:38,  3.44it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:40,  3.30it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:38,  3.44it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:30,  4.35it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:30,  4.35it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:30,  4.35it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:30,  4.33it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:30,  4.33it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:30,  4.33it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:29,  4.54it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:29,  4.54it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:29,  4.54it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:29,  4.54it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:29,  4.54it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:29,  4.54it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:36,  3.64it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:36,  3.64it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:36,  3.64it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:36,  3.64it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:36,  3.64it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:28,  4.61it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:28,  4.61it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:28,  4.61it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|███████   | 311/443 [02:35<00:30,  4.35it/s, Materializing param=model.layers.28.input_layernorm.weight] 
Loading weights:  70%|███████   | 311/443 [02:35<00:30,  4.35it/s, Materializing param=model.layers.28.input_layernorm.weight]
Loading weights:  70%|███████   | 312/443 [02:35<00:30,  4.35it/s, Materializing param=model.layers.28.mlp.down_proj.weight]  
Loading weights:  70%|███████   | 312/443 [02:35<00:30,  4.35it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  70%|███████   | 311/443 [02:35<00:30,  4.33it/s, Materializing param=model.layers.28.input_layernorm.weight] 
Loading weights:  70%|███████   | 311/443 [02:35<00:30,  4.33it/s, Materializing param=model.layers.28.input_layernorm.weight]
Loading weights:  70%|███████   | 312/443 [02:35<00:30,  4.33it/s, Materializing param=model.layers.28.mlp.down_proj.weight]  
Loading weights:  70%|███████   | 312/443 [02:35<00:30,  4.33it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  70%|███████   | 311/443 [02:35<00:29,  4.54it/s, Materializing param=model.layers.28.input_layernorm.weight] 
Loading weights:  70%|███████   | 311/443 [02:35<00:29,  4.54it/s, Materializing param=model.layers.28.input_layernorm.weight]
Loading weights:  70%|███████   | 312/443 [02:35<00:28,  4.54it/s, Materializing param=model.layers.28.mlp.down_proj.weight]  
Loading weights:  70%|███████   | 311/443 [02:35<00:36,  3.64it/s, Materializing param=model.layers.28.input_layernorm.weight] 
Loading weights:  70%|███████   | 312/443 [02:35<00:28,  4.54it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  70%|███████   | 311/443 [02:35<00:36,  3.64it/s, Materializing param=model.layers.28.input_layernorm.weight]
Loading weights:  70%|███████   | 311/443 [02:35<00:28,  4.61it/s, Materializing param=model.layers.28.input_layernorm.weight] 
Loading weights:  70%|███████   | 311/443 [02:35<00:28,  4.61it/s, Materializing param=model.layers.28.input_layernorm.weight]
Loading weights:  70%|███████   | 312/443 [02:35<00:35,  3.64it/s, Materializing param=model.layers.28.mlp.down_proj.weight]  
Loading weights:  70%|███████   | 312/443 [02:35<00:35,  3.64it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  70%|███████   | 312/443 [02:35<00:28,  4.61it/s, Materializing param=model.layers.28.mlp.down_proj.weight]  
Loading weights:  70%|███████   | 312/443 [02:35<00:28,  4.61it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:30,  4.42it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:30,  4.42it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:30,  4.42it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|███████   | 311/443 [02:35<00:29,  4.42it/s, Materializing param=model.layers.28.input_layernorm.weight] 
Loading weights:  70%|███████   | 311/443 [02:35<00:29,  4.42it/s, Materializing param=model.layers.28.input_layernorm.weight]
Loading weights:  70%|███████   | 312/443 [02:35<00:29,  4.42it/s, Materializing param=model.layers.28.mlp.down_proj.weight]  
Loading weights:  70%|███████   | 312/443 [02:35<00:29,  4.42it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:44,  3.02it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:44,  3.02it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 308/443 [02:35<00:44,  3.02it/s, Materializing param=model.layers.27.self_attn.q_norm.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:44,  3.02it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 309/443 [02:35<00:44,  3.02it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:44,  3.02it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|██████▉   | 310/443 [02:35<00:44,  3.02it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|███████   | 311/443 [02:35<00:43,  3.02it/s, Materializing param=model.layers.28.input_layernorm.weight] 
Loading weights:  70%|███████   | 311/443 [02:35<00:43,  3.02it/s, Materializing param=model.layers.28.input_layernorm.weight]
Loading weights:  70%|███████   | 312/443 [02:35<00:43,  3.02it/s, Materializing param=model.layers.28.mlp.down_proj.weight]  
Loading weights:  70%|███████   | 312/443 [02:35<00:43,  3.02it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:52,  2.47it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:56,  2.31it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:52,  2.47it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:56,  2.31it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:55,  2.34it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:50,  2.60it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:53,  2.44it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:52,  2.47it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:56,  2.31it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:55,  2.36it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:56,  2.31it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:55,  2.34it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:53,  2.44it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:50,  2.60it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:53,  2.44it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:50,  2.60it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:55,  2.34it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:37<00:52,  2.47it/s, Materializing param=model.layers.28.mlp.up_proj.weight]  
Loading weights:  71%|███████   | 313/443 [02:37<00:55,  2.36it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:37<00:52,  2.47it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:56,  2.31it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:55,  2.36it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:37<00:56,  2.31it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  70%|███████   | 311/443 [02:38<01:19,  1.66it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
Loading weights:  70%|███████   | 311/443 [02:38<01:19,  1.66it/s, Materializing param=model.layers.28.input_layernorm.weight] 
Loading weights:  70%|███████   | 311/443 [02:38<01:19,  1.66it/s, Materializing param=model.layers.28.input_layernorm.weight]
Loading weights:  70%|███████   | 312/443 [02:38<01:19,  1.66it/s, Materializing param=model.layers.28.mlp.down_proj.weight]  
Loading weights:  70%|███████   | 312/443 [02:38<01:19,  1.66it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:38<01:18,  1.66it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 313/443 [02:38<01:18,  1.66it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<00:54,  2.36it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<00:54,  2.36it/s, Materializing param=model.layers.28.mlp.up_proj.weight]  
Loading weights:  71%|███████   | 314/443 [02:38<00:54,  2.36it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<00:54,  2.38it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<00:54,  2.38it/s, Materializing param=model.layers.28.mlp.up_proj.weight]  
Loading weights:  71%|███████   | 314/443 [02:38<00:54,  2.38it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<01:00,  2.14it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<01:00,  2.14it/s, Materializing param=model.layers.28.mlp.up_proj.weight]  
Loading weights:  71%|███████   | 314/443 [02:38<01:00,  2.14it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<00:56,  2.26it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<01:03,  2.05it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<01:03,  2.05it/s, Materializing param=model.layers.28.mlp.up_proj.weight]  
Loading weights:  71%|███████   | 314/443 [02:38<00:56,  2.26it/s, Materializing param=model.layers.28.mlp.up_proj.weight]  
Loading weights:  71%|███████   | 314/443 [02:38<01:03,  2.05it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<00:56,  2.26it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<00:59,  2.15it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<00:59,  2.15it/s, Materializing param=model.layers.28.mlp.up_proj.weight]  
Loading weights:  71%|███████   | 314/443 [02:38<00:59,  2.15it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<00:58,  2.22it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
Loading weights:  71%|███████   | 314/443 [02:38<00:58,  2.22it/s, Materializing param=model.layers.28.mlp.up_proj.weight]  
Loading weights:  71%|███████   | 314/443 [02:38<00:58,  2.22it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:38<00:57,  2.22it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:38<00:57,  2.22it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████   | 315/443 [02:38<00:57,  2.22it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████▏  | 316/443 [02:38<00:57,  2.22it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]        
Loading weights:  71%|███████▏  | 316/443 [02:38<00:57,  2.22it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]
Loading weights:  72%|███████▏  | 317/443 [02:38<00:56,  2.22it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 317/443 [02:38<00:56,  2.22it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:38<00:54,  2.34it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:38<00:54,  2.34it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████   | 315/443 [02:38<00:54,  2.34it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████▏  | 316/443 [02:39<00:54,  2.34it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]        
Loading weights:  71%|███████▏  | 316/443 [02:39<00:54,  2.34it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<00:33,  3.71it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<00:53,  2.34it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<00:33,  3.71it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<00:53,  2.34it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<00:33,  3.71it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<00:53,  2.34it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<00:53,  2.34it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:01,  2.08it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:01,  2.08it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:01,  2.08it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████▏  | 316/443 [02:39<01:00,  2.08it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]        
Loading weights:  71%|███████▏  | 316/443 [02:39<01:00,  2.08it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<01:00,  2.08it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<01:00,  2.08it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<01:00,  2.08it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<01:00,  2.08it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:03,  2.02it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:03,  2.02it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:03,  2.02it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████▏  | 316/443 [02:39<01:02,  2.02it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]        
Loading weights:  71%|███████▏  | 316/443 [02:39<01:02,  2.02it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<01:02,  2.02it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<01:02,  2.02it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:02,  2.05it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:02,  2.05it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:02,  2.05it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████▏  | 316/443 [02:39<01:01,  2.05it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]        
Loading weights:  71%|███████▏  | 316/443 [02:39<01:01,  2.05it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<01:01,  2.05it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<01:01,  2.05it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<01:02,  2.02it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<01:02,  2.02it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<00:57,  2.23it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<00:57,  2.23it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████   | 315/443 [02:39<00:57,  2.23it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████▏  | 316/443 [02:39<00:57,  2.23it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]        
Loading weights:  71%|███████▏  | 316/443 [02:39<00:57,  2.23it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<00:56,  2.23it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<00:56,  2.23it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<00:58,  2.20it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<00:58,  2.20it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████   | 315/443 [02:39<00:58,  2.20it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<01:00,  2.05it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<01:00,  2.05it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  71%|███████▏  | 316/443 [02:39<00:57,  2.20it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]        
Loading weights:  71%|███████▏  | 316/443 [02:39<00:57,  2.20it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<00:57,  2.20it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<00:57,  2.20it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<00:56,  2.23it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<00:56,  2.23it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<00:56,  2.20it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<00:56,  2.20it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:01,  2.10it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:01,  2.10it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████   | 315/443 [02:39<01:01,  2.10it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
Loading weights:  71%|███████▏  | 316/443 [02:39<01:00,  2.10it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]        
Loading weights:  71%|███████▏  | 316/443 [02:39<01:00,  2.10it/s, Materializing param=model.layers.28.self_attn.k_norm.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<01:00,  2.10it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 317/443 [02:39<01:00,  2.10it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<00:59,  2.10it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 318/443 [02:39<00:59,  2.10it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:35,  3.51it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:36,  3.42it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:34,  3.58it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:34,  3.64it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:37,  3.32it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:39,  3.11it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:35,  3.51it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:34,  3.58it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:34,  3.64it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:35,  3.51it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:34,  3.58it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:37,  3.32it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:33,  3.71it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:39,  3.11it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:37,  3.32it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:34,  3.64it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:33,  3.71it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:39,  3.11it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:35,  3.51it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:35,  3.51it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:34,  3.56it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:33,  3.64it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:36,  3.42it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:34,  3.56it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:33,  3.64it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:34,  3.56it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:37,  3.32it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:37,  3.32it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:36,  3.42it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:35,  3.42it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:35,  3.42it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:35,  3.49it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:35,  3.49it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 319/443 [02:39<00:35,  3.49it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:35,  3.49it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:35,  3.49it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:34,  3.49it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:34,  3.49it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:39,  3.11it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:39,  3.11it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  73%|███████▎  | 322/443 [02:39<00:34,  3.49it/s, Materializing param=model.layers.29.input_layernorm.weight] 
Loading weights:  73%|███████▎  | 322/443 [02:39<00:34,  3.49it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|███████▎  | 323/443 [02:39<00:34,  3.49it/s, Materializing param=model.layers.29.mlp.down_proj.weight]  
Loading weights:  73%|███████▎  | 323/443 [02:39<00:34,  3.49it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:32,  3.79it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:32,  3.79it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:32,  3.79it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  73%|███████▎  | 322/443 [02:39<00:31,  3.79it/s, Materializing param=model.layers.29.input_layernorm.weight] 
Loading weights:  73%|███████▎  | 322/443 [02:39<00:31,  3.79it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|███████▎  | 323/443 [02:39<00:31,  3.79it/s, Materializing param=model.layers.29.mlp.down_proj.weight]  
Loading weights:  73%|███████▎  | 323/443 [02:39<00:31,  3.79it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:28,  4.29it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:29,  4.16it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:29,  4.16it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:28,  4.29it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:31,  3.89it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:29,  4.16it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:28,  4.35it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:31,  3.89it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:28,  4.35it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:31,  3.89it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:28,  4.35it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:28,  4.29it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:27,  4.49it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:27,  4.49it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:32,  3.82it/s, Materializing param=model.layers.28.self_attn.q_norm.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:27,  4.49it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:32,  3.82it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 320/443 [02:39<00:32,  3.82it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:31,  3.82it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  72%|███████▏  | 321/443 [02:39<00:31,  3.82it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
Loading weights:  73%|███████▎  | 322/443 [02:39<00:29,  4.16it/s, Materializing param=model.layers.29.input_layernorm.weight] 
Loading weights:  73%|███████▎  | 322/443 [02:39<00:29,  4.16it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|███████▎  | 323/443 [02:39<00:28,  4.16it/s, Materializing param=model.layers.29.mlp.down_proj.weight]  
Loading weights:  73%|███████▎  | 323/443 [02:39<00:28,  4.16it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 322/443 [02:39<00:31,  3.89it/s, Materializing param=model.layers.29.input_layernorm.weight] 
Loading weights:  73%|███████▎  | 322/443 [02:39<00:31,  3.89it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|███████▎  | 323/443 [02:39<00:30,  3.89it/s, Materializing param=model.layers.29.mlp.down_proj.weight]  
Loading weights:  73%|███████▎  | 322/443 [02:39<00:28,  4.29it/s, Materializing param=model.layers.29.input_layernorm.weight] 
Loading weights:  73%|███████▎  | 323/443 [02:39<00:30,  3.89it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 322/443 [02:39<00:28,  4.29it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|███████▎  | 322/443 [02:39<00:27,  4.35it/s, Materializing param=model.layers.29.input_layernorm.weight] 
Loading weights:  73%|███████▎  | 322/443 [02:39<00:27,  4.35it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|███████▎  | 323/443 [02:39<00:27,  4.35it/s, Materializing param=model.layers.29.mlp.down_proj.weight]  
Loading weights:  73%|███████▎  | 323/443 [02:39<00:27,  4.35it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 322/443 [02:39<00:26,  4.49it/s, Materializing param=model.layers.29.input_layernorm.weight] 
Loading weights:  73%|███████▎  | 322/443 [02:39<00:26,  4.49it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|███████▎  | 323/443 [02:39<00:26,  4.49it/s, Materializing param=model.layers.29.mlp.down_proj.weight]  
Loading weights:  73%|███████▎  | 323/443 [02:39<00:26,  4.49it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 322/443 [02:39<00:31,  3.82it/s, Materializing param=model.layers.29.input_layernorm.weight] 
Loading weights:  73%|███████▎  | 322/443 [02:39<00:31,  3.82it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:46,  2.58it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:43,  2.76it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:42,  2.80it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:40,  2.91it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:41,  2.87it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 323/443 [02:41<00:48,  2.48it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:38,  3.08it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:42,  2.80it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:46,  2.58it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:40,  2.91it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:41,  2.87it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:42,  2.80it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:46,  2.58it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:40,  2.91it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:41,  2.87it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 323/443 [02:41<00:45,  2.66it/s, Materializing param=model.layers.29.input_layernorm.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:38,  3.08it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:38,  3.08it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 323/443 [02:41<00:48,  2.48it/s, Materializing param=model.layers.29.mlp.down_proj.weight]  
Loading weights:  73%|███████▎  | 323/443 [02:41<00:45,  2.66it/s, Materializing param=model.layers.29.mlp.down_proj.weight]  
Loading weights:  73%|███████▎  | 323/443 [02:41<00:45,  2.66it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:44,  2.66it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:44,  2.66it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 323/443 [02:41<00:48,  2.48it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:47,  2.48it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:47,  2.48it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:43,  2.76it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 324/443 [02:41<00:43,  2.76it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:42,  2.76it/s, Materializing param=model.layers.29.mlp.up_proj.weight]  
Loading weights:  73%|███████▎  | 325/443 [02:41<00:42,  2.76it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:40,  2.95it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:40,  2.95it/s, Materializing param=model.layers.29.mlp.up_proj.weight]  
Loading weights:  73%|███████▎  | 325/443 [02:41<00:40,  2.95it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:41,  2.82it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:44,  2.68it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:42,  2.77it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:39,  2.96it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:44,  2.68it/s, Materializing param=model.layers.29.mlp.up_proj.weight]  
Loading weights:  73%|███████▎  | 325/443 [02:41<00:47,  2.50it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:43,  2.74it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:39,  2.96it/s, Materializing param=model.layers.29.mlp.up_proj.weight]  
Loading weights:  73%|███████▎  | 325/443 [02:41<00:44,  2.68it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:42,  2.77it/s, Materializing param=model.layers.29.mlp.up_proj.weight]  
Loading weights:  73%|███████▎  | 325/443 [02:41<00:39,  2.96it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:42,  2.77it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:43,  2.74it/s, Materializing param=model.layers.29.mlp.up_proj.weight]  
Loading weights:  73%|███████▎  | 325/443 [02:41<00:43,  2.74it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:41,  2.82it/s, Materializing param=model.layers.29.mlp.up_proj.weight]  
Loading weights:  73%|███████▎  | 325/443 [02:41<00:41,  2.82it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  73%|███████▎  | 325/443 [02:41<00:47,  2.50it/s, Materializing param=model.layers.29.mlp.up_proj.weight]  
Loading weights:  73%|███████▎  | 325/443 [02:41<00:47,  2.50it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:45,  2.56it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:45,  2.56it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:45,  2.56it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▍  | 327/443 [02:42<00:45,  2.56it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]        
Loading weights:  74%|███████▍  | 327/443 [02:42<00:45,  2.56it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:44,  2.56it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:44,  2.56it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:44,  2.56it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:44,  2.56it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:44,  2.63it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:44,  2.63it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:44,  2.63it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▍  | 327/443 [02:42<00:44,  2.63it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]        
Loading weights:  74%|███████▍  | 327/443 [02:42<00:44,  2.63it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:43,  2.63it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:43,  2.63it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:43,  2.63it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:43,  2.63it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:52,  2.23it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:52,  2.23it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:52,  2.23it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▍  | 327/443 [02:42<00:51,  2.23it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]        
Loading weights:  74%|███████▍  | 327/443 [02:42<00:51,  2.23it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:51,  2.23it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:51,  2.23it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:49,  2.37it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:49,  2.37it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:49,  2.37it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▍  | 327/443 [02:42<00:48,  2.37it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]        
Loading weights:  74%|███████▍  | 327/443 [02:42<00:48,  2.37it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:48,  2.37it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:48,  2.37it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:48,  2.37it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:48,  2.37it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:48,  2.42it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:48,  2.42it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:48,  2.42it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▍  | 327/443 [02:42<00:47,  2.42it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]        
Loading weights:  74%|███████▍  | 327/443 [02:42<00:47,  2.42it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:47,  2.42it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:47,  2.42it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:47,  2.42it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:47,  2.42it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:51,  2.28it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:51,  2.28it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:51,  2.28it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▍  | 327/443 [02:42<00:50,  2.28it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]        
Loading weights:  74%|███████▍  | 327/443 [02:42<00:50,  2.28it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:51,  2.26it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:51,  2.26it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:51,  2.26it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▍  | 327/443 [02:42<00:51,  2.26it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]        
Loading weights:  74%|███████▍  | 327/443 [02:42<00:51,  2.26it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:48,  2.41it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:48,  2.41it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▎  | 326/443 [02:42<00:48,  2.41it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
Loading weights:  74%|███████▍  | 327/443 [02:42<00:48,  2.41it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]        
Loading weights:  74%|███████▍  | 327/443 [02:42<00:48,  2.41it/s, Materializing param=model.layers.29.self_attn.k_norm.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:47,  2.41it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:47,  2.41it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:50,  2.28it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:50,  2.28it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:50,  2.28it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:50,  2.28it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:50,  2.26it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 328/443 [02:42<00:50,  2.26it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:50,  2.26it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:50,  2.26it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:47,  2.41it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:47,  2.41it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:26,  4.21it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:30,  3.71it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:26,  4.21it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:30,  3.71it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:30,  3.71it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:26,  4.21it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:26,  4.22it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:42<00:26,  4.21it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:26,  4.22it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:26,  4.22it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  75%|███████▍  | 331/443 [02:42<00:26,  4.21it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:32,  3.55it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:32,  3.55it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 329/443 [02:42<00:32,  3.55it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:31,  3.55it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:31,  3.55it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:26,  4.24it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:26,  4.24it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:42<00:26,  4.24it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  75%|███████▍  | 331/443 [02:42<00:26,  4.24it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:42<00:26,  4.24it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:42<00:24,  4.55it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  75%|███████▍  | 331/443 [02:42<00:25,  4.37it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  75%|███████▍  | 331/443 [02:42<00:25,  4.37it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:42<00:24,  4.55it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:42<00:24,  4.55it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:42<00:25,  4.37it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:42<00:24,  4.55it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:42<00:25,  4.37it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:42<00:24,  4.55it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:42<00:25,  4.37it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▌  | 333/443 [02:43<00:25,  4.37it/s, Materializing param=model.layers.30.input_layernorm.weight] 
Loading weights:  75%|███████▌  | 333/443 [02:43<00:25,  4.37it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  75%|███████▌  | 334/443 [02:43<00:24,  4.37it/s, Materializing param=model.layers.30.mlp.down_proj.weight]  
Loading weights:  75%|███████▌  | 333/443 [02:43<00:24,  4.55it/s, Materializing param=model.layers.30.input_layernorm.weight] 
Loading weights:  75%|███████▌  | 334/443 [02:43<00:24,  4.37it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  75%|███████▌  | 333/443 [02:43<00:24,  4.55it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  75%|███████▌  | 334/443 [02:43<00:23,  4.55it/s, Materializing param=model.layers.30.mlp.down_proj.weight]  
Loading weights:  75%|███████▌  | 334/443 [02:43<00:23,  4.55it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  74%|███████▍  | 330/443 [02:43<00:28,  3.93it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 330/443 [02:43<00:28,  3.93it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:43<00:28,  3.93it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:43<00:28,  3.90it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 330/443 [02:43<00:27,  4.04it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
Loading weights:  74%|███████▍  | 330/443 [02:43<00:27,  4.04it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:43<00:27,  4.04it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:43<00:28,  3.90it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  74%|███████▍  | 330/443 [02:43<00:28,  3.90it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  75%|███████▍  | 331/443 [02:43<00:27,  4.04it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:43<00:27,  4.04it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:43<00:28,  3.90it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:43<00:28,  3.90it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:22,  4.89it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:43<00:29,  3.80it/s, Materializing param=model.layers.29.self_attn.q_norm.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:22,  4.89it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:43<00:29,  3.80it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:22,  4.89it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:43<00:29,  3.80it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:29,  3.80it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:29,  3.80it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▌  | 333/443 [02:43<00:28,  3.80it/s, Materializing param=model.layers.30.input_layernorm.weight] 
Loading weights:  75%|███████▌  | 333/443 [02:43<00:28,  3.80it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  75%|███████▌  | 334/443 [02:43<00:28,  3.80it/s, Materializing param=model.layers.30.mlp.down_proj.weight]  
Loading weights:  75%|███████▌  | 334/443 [02:43<00:28,  3.80it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  75%|███████▌  | 333/443 [02:43<00:22,  4.89it/s, Materializing param=model.layers.30.input_layernorm.weight] 
Loading weights:  75%|███████▌  | 333/443 [02:43<00:22,  4.89it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  75%|███████▌  | 334/443 [02:43<00:22,  4.89it/s, Materializing param=model.layers.30.mlp.down_proj.weight]  
Loading weights:  75%|███████▌  | 334/443 [02:43<00:22,  4.89it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:22,  4.93it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:22,  4.93it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:22,  4.93it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▌  | 333/443 [02:43<00:22,  4.93it/s, Materializing param=model.layers.30.input_layernorm.weight] 
Loading weights:  75%|███████▌  | 333/443 [02:43<00:22,  4.93it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:28,  3.90it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:28,  3.90it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:27,  4.04it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:27,  4.04it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:43<00:28,  3.93it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 331/443 [02:43<00:28,  3.93it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:28,  3.93it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▍  | 332/443 [02:43<00:28,  3.93it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
Loading weights:  75%|███████▌  | 333/443 [02:43<00:28,  3.90it/s, Materializing param=model.layers.30.input_layernorm.weight] 
Loading weights:  75%|███████▌  | 333/443 [02:43<00:28,  3.90it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  75%|███████▌  | 334/443 [02:43<00:27,  3.90it/s, Materializing param=model.layers.30.mlp.down_proj.weight]  
Loading weights:  75%|███████▌  | 334/443 [02:43<00:27,  3.90it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  75%|███████▌  | 333/443 [02:43<00:27,  3.93it/s, Materializing param=model.layers.30.input_layernorm.weight] 
Loading weights:  75%|███████▌  | 333/443 [02:43<00:27,  3.93it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  75%|███████▌  | 333/443 [02:43<00:27,  4.04it/s, Materializing param=model.layers.30.input_layernorm.weight] 
Loading weights:  75%|███████▌  | 333/443 [02:43<00:27,  4.04it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  75%|███████▌  | 334/443 [02:43<00:27,  3.93it/s, Materializing param=model.layers.30.mlp.down_proj.weight]  
Loading weights:  75%|███████▌  | 334/443 [02:43<00:27,  3.93it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:35,  3.00it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:37,  2.86it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:37,  2.91it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  75%|███████▌  | 334/443 [02:44<00:42,  2.58it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:33,  3.26it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  75%|███████▌  | 334/443 [02:44<00:36,  3.01it/s, Materializing param=model.layers.30.input_layernorm.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:33,  3.25it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:35,  3.00it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:36,  3.00it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:37,  2.86it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:33,  3.26it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:35,  3.00it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:37,  2.91it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:37,  2.86it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:33,  3.25it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:33,  3.26it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:36,  3.00it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:33,  3.25it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:37,  2.91it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  75%|███████▌  | 334/443 [02:44<00:36,  3.01it/s, Materializing param=model.layers.30.mlp.down_proj.weight]  
Loading weights:  76%|███████▌  | 335/443 [02:44<00:36,  3.00it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  75%|███████▌  | 334/443 [02:44<00:36,  3.01it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  75%|███████▌  | 334/443 [02:44<00:42,  2.58it/s, Materializing param=model.layers.30.mlp.down_proj.weight]  
Loading weights:  76%|███████▌  | 335/443 [02:44<00:35,  3.01it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:35,  3.01it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  75%|███████▌  | 334/443 [02:44<00:42,  2.58it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:41,  2.58it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 335/443 [02:44<00:41,  2.58it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<01:00,  1.77it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:59,  1.81it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:59,  1.79it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<01:02,  1.71it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:51,  2.09it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:53,  1.99it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:53,  1.99it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:59,  1.80it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<01:00,  1.77it/s, Materializing param=model.layers.30.mlp.up_proj.weight]  
Loading weights:  76%|███████▌  | 336/443 [02:46<01:02,  1.71it/s, Materializing param=model.layers.30.mlp.up_proj.weight]  
Loading weights:  76%|███████▌  | 336/443 [02:46<00:59,  1.79it/s, Materializing param=model.layers.30.mlp.up_proj.weight]  
Loading weights:  76%|███████▌  | 336/443 [02:46<01:00,  1.77it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:53,  1.99it/s, Materializing param=model.layers.30.mlp.up_proj.weight]  
Loading weights:  76%|███████▌  | 336/443 [02:46<00:51,  2.09it/s, Materializing param=model.layers.30.mlp.up_proj.weight]  
Loading weights:  76%|███████▌  | 336/443 [02:46<00:53,  1.99it/s, Materializing param=model.layers.30.mlp.up_proj.weight]  
Loading weights:  76%|███████▌  | 336/443 [02:46<01:02,  1.71it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:59,  1.79it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:59,  1.80it/s, Materializing param=model.layers.30.mlp.up_proj.weight]  
Loading weights:  76%|███████▌  | 336/443 [02:46<00:53,  1.99it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:51,  2.09it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:53,  1.99it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:59,  1.80it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 336/443 [02:46<00:59,  1.81it/s, Materializing param=model.layers.30.mlp.up_proj.weight]  
Loading weights:  76%|███████▌  | 336/443 [02:46<00:59,  1.81it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:31,  1.16it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:27,  1.21it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:27,  1.21it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:27,  1.21it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:31,  1.16it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:28,  1.19it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:31,  1.16it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:19,  1.34it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:27,  1.21it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:28,  1.19it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:15,  1.40it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:19,  1.34it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:28,  1.19it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:15,  1.40it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:19,  1.34it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:26,  1.21it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]        
Loading weights:  76%|███████▌  | 337/443 [02:48<01:15,  1.40it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:30,  1.16it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]        
Loading weights:  76%|███████▌  | 337/443 [02:48<01:19,  1.34it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:26,  1.21it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:27,  1.19it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]        
Loading weights:  76%|███████▋  | 338/443 [02:48<01:30,  1.16it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:25,  1.23it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:19,  1.34it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:27,  1.19it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:14,  1.40it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]        
Loading weights:  76%|███████▌  | 337/443 [02:48<01:19,  1.34it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:18,  1.34it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]        
Loading weights:  76%|███████▌  | 337/443 [02:48<01:25,  1.23it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:14,  1.40it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:25,  1.21it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:29,  1.16it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:25,  1.23it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:29,  1.16it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:25,  1.21it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:18,  1.34it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]        
Loading weights:  76%|███████▋  | 338/443 [02:48<01:18,  1.34it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:25,  1.23it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]        
Loading weights:  76%|███████▌  | 337/443 [02:48<01:27,  1.21it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:18,  1.34it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:25,  1.23it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:17,  1.34it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:14,  1.40it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:17,  1.34it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:24,  1.23it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:17,  1.34it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:14,  1.40it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  76%|███████▌  | 337/443 [02:48<01:27,  1.21it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:24,  1.23it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:17,  1.34it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  76%|███████▋  | 338/443 [02:48<01:26,  1.21it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]        
Loading weights:  76%|███████▋  | 338/443 [02:48<01:26,  1.21it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:25,  1.21it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 339/443 [02:48<01:25,  1.21it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 339/443 [02:49<01:00,  1.71it/s, Materializing param=model.layers.30.self_attn.k_norm.weight]
Loading weights:  77%|███████▋  | 339/443 [02:49<01:00,  1.71it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 339/443 [02:49<01:00,  1.71it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:51,  2.00it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<01:00,  1.71it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:51,  2.00it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:48,  2.10it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<01:00,  1.71it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:51,  2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:52,  1.95it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:51,  2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:48,  2.10it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:51,  2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:51,  2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:48,  2.10it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:51,  2.01it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:47,  2.16it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:47,  2.16it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:47,  2.16it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:51,  2.01it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:51,  2.01it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:52,  1.95it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:52,  1.95it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:49,  2.10it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:49,  2.10it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 340/443 [02:49<00:49,  2.10it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:50,  2.01it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:48,  2.10it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:50,  2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:48,  2.09it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:48,  2.09it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:50,  2.00it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:48,  2.10it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:50,  2.00it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:48,  2.09it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:50,  2.00it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:48,  2.10it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:47,  2.14it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:50,  2.00it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:47,  2.14it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:50,  2.00it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:47,  2.14it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:50,  2.00it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:50,  2.00it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:47,  2.14it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:48,  2.10it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:47,  2.14it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:48,  2.10it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:50,  2.01it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:50,  2.00it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:50,  2.00it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:43,  2.34it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:43,  2.34it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:43,  2.34it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:48,  2.09it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:48,  2.09it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:48,  2.09it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:48,  2.09it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:51,  1.97it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:48,  2.09it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:48,  2.09it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:48,  2.09it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:47,  2.09it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:47,  2.09it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:51,  1.97it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 341/443 [02:49<00:51,  1.97it/s, Materializing param=model.layers.30.self_attn.q_norm.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:51,  1.97it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 342/443 [02:49<00:51,  1.97it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:50,  1.97it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:50,  1.97it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  78%|███████▊  | 344/443 [02:49<00:30,  3.28it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  78%|███████▊  | 344/443 [02:49<00:31,  3.19it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  78%|███████▊  | 344/443 [02:49<00:31,  3.19it/s, Materializing param=model.layers.31.input_layernorm.weight] 
Loading weights:  78%|███████▊  | 344/443 [02:49<00:30,  3.28it/s, Materializing param=model.layers.31.input_layernorm.weight] 
Loading weights:  78%|███████▊  | 344/443 [02:49<00:31,  3.19it/s, Materializing param=model.layers.31.input_layernorm.weight]
Loading weights:  78%|███████▊  | 344/443 [02:49<00:30,  3.28it/s, Materializing param=model.layers.31.input_layernorm.weight]
Loading weights:  78%|███████▊  | 345/443 [02:49<00:30,  3.19it/s, Materializing param=model.layers.31.mlp.down_proj.weight]  
Loading weights:  78%|███████▊  | 345/443 [02:49<00:29,  3.28it/s, Materializing param=model.layers.31.mlp.down_proj.weight]  
Loading weights:  78%|███████▊  | 345/443 [02:49<00:30,  3.19it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 345/443 [02:49<00:29,  3.28it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:35,  2.85it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:35,  2.85it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:35,  2.85it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:38,  2.63it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:38,  2.63it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:38,  2.63it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:35,  2.81it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:35,  2.81it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:35,  2.81it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  78%|███████▊  | 344/443 [02:49<00:34,  2.85it/s, Materializing param=model.layers.31.input_layernorm.weight] 
Loading weights:  78%|███████▊  | 344/443 [02:49<00:34,  2.85it/s, Materializing param=model.layers.31.input_layernorm.weight]
Loading weights:  78%|███████▊  | 345/443 [02:49<00:34,  2.85it/s, Materializing param=model.layers.31.mlp.down_proj.weight]  
Loading weights:  78%|███████▊  | 345/443 [02:49<00:34,  2.85it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:36,  2.70it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:36,  2.70it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:36,  2.70it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:35,  2.83it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:36,  2.72it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:35,  2.83it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:36,  2.72it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:35,  2.83it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  77%|███████▋  | 343/443 [02:49<00:36,  2.72it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
Loading weights:  78%|███████▊  | 344/443 [02:49<00:37,  2.63it/s, Materializing param=model.layers.31.input_layernorm.weight] 
Loading weights:  78%|███████▊  | 344/443 [02:49<00:37,  2.63it/s, Materializing param=model.layers.31.input_layernorm.weight]
Loading weights:  78%|███████▊  | 344/443 [02:49<00:35,  2.81it/s, Materializing param=model.layers.31.input_layernorm.weight] 
Loading weights:  78%|███████▊  | 344/443 [02:49<00:35,  2.81it/s, Materializing param=model.layers.31.input_layernorm.weight]
Loading weights:  78%|███████▊  | 345/443 [02:49<00:34,  2.81it/s, Materializing param=model.layers.31.mlp.down_proj.weight]  
Loading weights:  78%|███████▊  | 345/443 [02:49<00:34,  2.81it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 344/443 [02:49<00:36,  2.70it/s, Materializing param=model.layers.31.input_layernorm.weight] 
Loading weights:  78%|███████▊  | 344/443 [02:49<00:36,  2.70it/s, Materializing param=model.layers.31.input_layernorm.weight]
Loading weights:  78%|███████▊  | 345/443 [02:49<00:36,  2.70it/s, Materializing param=model.layers.31.mlp.down_proj.weight]  
Loading weights:  78%|███████▊  | 345/443 [02:49<00:36,  2.70it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 344/443 [02:49<00:34,  2.83it/s, Materializing param=model.layers.31.input_layernorm.weight] 
Loading weights:  78%|███████▊  | 344/443 [02:49<00:34,  2.83it/s, Materializing param=model.layers.31.input_layernorm.weight]
Loading weights:  78%|███████▊  | 344/443 [02:49<00:36,  2.72it/s, Materializing param=model.layers.31.input_layernorm.weight] 
Loading weights:  78%|███████▊  | 344/443 [02:49<00:36,  2.72it/s, Materializing param=model.layers.31.input_layernorm.weight]
Loading weights:  78%|███████▊  | 345/443 [02:49<00:34,  2.83it/s, Materializing param=model.layers.31.mlp.down_proj.weight]  
Loading weights:  78%|███████▊  | 345/443 [02:49<00:36,  2.72it/s, Materializing param=model.layers.31.mlp.down_proj.weight]  
Loading weights:  78%|███████▊  | 345/443 [02:49<00:34,  2.83it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 345/443 [02:49<00:36,  2.72it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:47,  2.03it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:51,  1.90it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:48,  2.00it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:47,  2.03it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:48,  2.00it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:49,  1.95it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:47,  2.06it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:47,  2.03it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:48,  2.00it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:51,  1.90it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:47,  2.03it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:48,  2.00it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:47,  2.03it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:49,  1.95it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:47,  2.06it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:48,  2.00it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:51,  1.90it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:47,  2.03it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:49,  1.95it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:47,  2.06it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:51<00:48,  2.00it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:13,  1.31it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:16,  1.25it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:11,  1.34it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:12,  1.32it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:14,  1.28it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:13,  1.31it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 345/443 [02:53<01:32,  1.06it/s, Materializing param=model.layers.31.input_layernorm.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:13,  1.31it/s, Materializing param=model.layers.31.mlp.up_proj.weight]  
Loading weights:  78%|███████▊  | 347/443 [02:53<01:11,  1.34it/s, Materializing param=model.layers.31.mlp.up_proj.weight]  
Loading weights:  78%|███████▊  | 347/443 [02:53<01:16,  1.25it/s, Materializing param=model.layers.31.mlp.up_proj.weight]  
Loading weights:  78%|███████▊  | 347/443 [02:53<01:13,  1.31it/s, Materializing param=model.layers.31.mlp.up_proj.weight]  
Loading weights:  78%|███████▊  | 347/443 [02:53<01:12,  1.32it/s, Materializing param=model.layers.31.mlp.up_proj.weight]  
Loading weights:  78%|███████▊  | 347/443 [02:53<01:14,  1.28it/s, Materializing param=model.layers.31.mlp.up_proj.weight]  
Loading weights:  78%|███████▊  | 347/443 [02:53<01:16,  1.25it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:13,  1.31it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:13,  1.31it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:11,  1.34it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:12,  1.32it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:14,  1.28it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:12,  1.33it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:12,  1.33it/s, Materializing param=model.layers.31.mlp.up_proj.weight]  
Loading weights:  78%|███████▊  | 345/443 [02:53<01:32,  1.06it/s, Materializing param=model.layers.31.mlp.down_proj.weight]  
Loading weights:  78%|███████▊  | 347/443 [02:53<01:12,  1.33it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  78%|███████▊  | 345/443 [02:53<01:32,  1.06it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
Loading weights:  79%|███████▊  | 348/443 [02:53<01:11,  1.33it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▊  | 348/443 [02:53<01:11,  1.33it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▉  | 349/443 [02:53<01:10,  1.33it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]        
Loading weights:  78%|███████▊  | 346/443 [02:53<01:31,  1.06it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  79%|███████▉  | 349/443 [02:53<01:10,  1.33it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]
Loading weights:  79%|███████▉  | 350/443 [02:53<01:09,  1.33it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▉  | 350/443 [02:53<01:09,  1.33it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  78%|███████▊  | 346/443 [02:53<01:31,  1.06it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
Loading weights:  78%|███████▊  | 347/443 [02:53<01:30,  1.06it/s, Materializing param=model.layers.31.mlp.up_proj.weight]  
Loading weights:  78%|███████▊  | 347/443 [02:53<01:30,  1.06it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:09,  1.33it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:09,  1.33it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.55it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.55it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.55it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  80%|███████▉  | 353/443 [02:54<00:35,  2.55it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 353/443 [02:54<00:35,  2.55it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:00,  1.58it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:00,  1.58it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:00,  1.58it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▉  | 349/443 [02:54<00:59,  1.58it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]        
Loading weights:  79%|███████▉  | 349/443 [02:54<00:59,  1.58it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:09,  1.37it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:09,  1.37it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:09,  1.37it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▉  | 349/443 [02:54<01:08,  1.37it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]        
Loading weights:  79%|███████▉  | 349/443 [02:54<01:08,  1.37it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<00:58,  1.58it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<00:58,  1.58it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<00:58,  1.58it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<00:58,  1.58it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<01:07,  1.37it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<01:07,  1.37it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:06,  1.37it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:06,  1.37it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:54<00:29,  2.98it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:54<00:29,  2.98it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:54<00:29,  2.98it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:33,  2.70it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:33,  2.70it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:33,  2.70it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  80%|███████▉  | 353/443 [02:54<00:33,  2.70it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 353/443 [02:54<00:33,  2.70it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:12,  1.31it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:12,  1.31it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:11,  1.33it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:12,  1.31it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▉  | 349/443 [02:54<01:11,  1.31it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]        
Loading weights:  79%|███████▊  | 348/443 [02:54<01:11,  1.33it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▉  | 349/443 [02:54<01:11,  1.31it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:11,  1.33it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<01:11,  1.31it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▉  | 349/443 [02:54<01:10,  1.33it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]        
Loading weights:  79%|███████▉  | 350/443 [02:54<01:11,  1.31it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▉  | 349/443 [02:54<01:10,  1.33it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<01:09,  1.33it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<01:09,  1.33it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:10,  1.34it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:10,  1.34it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:10,  1.34it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▉  | 349/443 [02:54<01:10,  1.34it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]        
Loading weights:  79%|███████▉  | 349/443 [02:54<01:10,  1.34it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:10,  1.35it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<01:09,  1.34it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:14,  1.28it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<01:09,  1.34it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:14,  1.28it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:10,  1.35it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:14,  1.28it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▊  | 348/443 [02:54<01:10,  1.35it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
Loading weights:  79%|███████▉  | 349/443 [02:54<01:13,  1.28it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]        
Loading weights:  79%|███████▉  | 349/443 [02:54<01:13,  1.28it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]
Loading weights:  79%|███████▉  | 349/443 [02:54<01:09,  1.35it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]        
Loading weights:  79%|███████▉  | 350/443 [02:54<01:12,  1.28it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▉  | 349/443 [02:54<01:09,  1.35it/s, Materializing param=model.layers.31.self_attn.k_norm.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<01:12,  1.28it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<01:08,  1.35it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  79%|███████▉  | 350/443 [02:54<01:08,  1.35it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
Loading weights:  80%|████████  | 355/443 [02:54<00:29,  2.98it/s, Materializing param=model.layers.32.input_layernorm.weight] 
Loading weights:  80%|████████  | 355/443 [02:54<00:29,  2.98it/s, Materializing param=model.layers.32.input_layernorm.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:08,  1.34it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:08,  1.34it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:11,  1.28it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:11,  1.28it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:09,  1.33it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:10,  1.31it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:09,  1.33it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:10,  1.31it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:07,  1.35it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 351/443 [02:54<01:07,  1.35it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:34,  2.66it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:34,  2.66it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:34,  2.66it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  80%|███████▉  | 353/443 [02:54<00:33,  2.66it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 353/443 [02:54<00:33,  2.66it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:54<00:26,  3.30it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:54<00:26,  3.30it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:54<00:26,  3.30it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|████████  | 355/443 [02:54<00:26,  3.30it/s, Materializing param=model.layers.32.input_layernorm.weight] 
Loading weights:  80%|████████  | 355/443 [02:54<00:26,  3.30it/s, Materializing param=model.layers.32.input_layernorm.weight]
Loading weights:  80%|████████  | 356/443 [02:54<00:26,  3.30it/s, Materializing param=model.layers.32.mlp.down_proj.weight]  
Loading weights:  80%|████████  | 356/443 [02:54<00:26,  3.30it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.58it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.58it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.58it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  80%|███████▉  | 353/443 [02:54<00:34,  2.58it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 353/443 [02:54<00:34,  2.58it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.57it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.57it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.57it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:36,  2.50it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:36,  2.50it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:36,  2.50it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  80%|███████▉  | 354/443 [02:54<00:26,  3.36it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:54<00:26,  3.36it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:54<00:26,  3.36it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:36,  2.51it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:36,  2.51it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:36,  2.51it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.53it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.53it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  79%|███████▉  | 352/443 [02:54<00:35,  2.53it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  80%|███████▉  | 353/443 [02:54<00:35,  2.53it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 353/443 [02:54<00:35,  2.53it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|████████  | 355/443 [02:54<00:26,  3.36it/s, Materializing param=model.layers.32.input_layernorm.weight] 
Loading weights:  80%|████████  | 355/443 [02:54<00:26,  3.36it/s, Materializing param=model.layers.32.input_layernorm.weight]
Loading weights:  80%|████████  | 356/443 [02:54<00:25,  3.36it/s, Materializing param=model.layers.32.mlp.down_proj.weight]  
Loading weights:  80%|████████  | 356/443 [02:54<00:25,  3.36it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:27,  3.27it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:27,  3.27it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:27,  3.27it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|███████▉  | 353/443 [02:55<00:32,  2.77it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  80%|███████▉  | 353/443 [02:55<00:32,  2.77it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 353/443 [02:55<00:32,  2.77it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:32,  2.77it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:32,  2.77it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:27,  3.23it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:27,  3.23it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:27,  3.23it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|███████▉  | 353/443 [02:55<00:32,  2.77it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  80%|███████▉  | 353/443 [02:55<00:32,  2.77it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 353/443 [02:55<00:32,  2.77it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:32,  2.77it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:32,  2.77it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|████████  | 355/443 [02:55<00:26,  3.27it/s, Materializing param=model.layers.32.input_layernorm.weight] 
Loading weights:  80%|████████  | 355/443 [02:55<00:26,  3.27it/s, Materializing param=model.layers.32.input_layernorm.weight]
Loading weights:  80%|████████  | 356/443 [02:55<00:26,  3.27it/s, Materializing param=model.layers.32.mlp.down_proj.weight]  
Loading weights:  80%|████████  | 356/443 [02:55<00:26,  3.27it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  80%|████████  | 355/443 [02:55<00:31,  2.77it/s, Materializing param=model.layers.32.input_layernorm.weight] 
Loading weights:  80%|████████  | 355/443 [02:55<00:31,  2.77it/s, Materializing param=model.layers.32.input_layernorm.weight]
Loading weights:  80%|████████  | 356/443 [02:55<00:31,  2.77it/s, Materializing param=model.layers.32.mlp.down_proj.weight]  
Loading weights:  80%|████████  | 356/443 [02:55<00:31,  2.77it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  80%|███████▉  | 353/443 [02:55<00:32,  2.79it/s, Materializing param=model.layers.31.self_attn.q_norm.weight]
Loading weights:  80%|███████▉  | 353/443 [02:55<00:32,  2.79it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 353/443 [02:55<00:32,  2.79it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:31,  2.79it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|███████▉  | 354/443 [02:55<00:31,  2.79it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
Loading weights:  80%|████████  | 355/443 [02:55<00:31,  2.79it/s, Materializing param=model.layers.32.input_layernorm.weight] 
Loading weights:  80%|████████  | 355/443 [02:55<00:31,  2.79it/s, Materializing param=model.layers.32.input_layernorm.weight]
Loading weights:  80%|████████  | 356/443 [02:55<00:31,  2.79it/s, Materializing param=model.layers.32.mlp.down_proj.weight]  
Loading weights:  80%|████████  | 356/443 [02:55<00:31,  2.79it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  80%|████████  | 355/443 [02:55<00:31,  2.77it/s, Materializing param=model.layers.32.input_layernorm.weight] 
Loading weights:  80%|████████  | 355/443 [02:55<00:27,  3.23it/s, Materializing param=model.layers.32.input_layernorm.weight] 
Loading weights:  80%|████████  | 355/443 [02:55<00:27,  3.23it/s, Materializing param=model.layers.32.input_layernorm.weight]
Loading weights:  80%|████████  | 355/443 [02:55<00:31,  2.77it/s, Materializing param=model.layers.32.input_layernorm.weight]
Loading weights:  80%|████████  | 356/443 [02:55<00:26,  3.23it/s, Materializing param=model.layers.32.mlp.down_proj.weight]  
Loading weights:  80%|████████  | 356/443 [02:55<00:26,  3.23it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  80%|████████  | 356/443 [02:55<00:31,  2.77it/s, Materializing param=model.layers.32.mlp.down_proj.weight]  
Loading weights:  80%|████████  | 356/443 [02:55<00:31,  2.77it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:35,  2.42it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:34,  2.51it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:33,  2.58it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:34,  2.51it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:35,  2.43it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:33,  2.60it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  80%|████████  | 356/443 [02:56<00:44,  1.95it/s, Materializing param=model.layers.32.input_layernorm.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:33,  2.58it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:32,  2.62it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:35,  2.43it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:34,  2.51it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:34,  2.51it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:33,  2.58it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:33,  2.60it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:35,  2.43it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:34,  2.51it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  80%|████████  | 356/443 [02:56<00:44,  1.95it/s, Materializing param=model.layers.32.mlp.down_proj.weight]  
Loading weights:  81%|████████  | 357/443 [02:56<00:33,  2.60it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:34,  2.51it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:32,  2.62it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  80%|████████  | 356/443 [02:56<00:44,  1.95it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:35,  2.42it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:56<00:32,  2.60it/s, Materializing param=model.layers.32.mlp.up_proj.weight]  
Loading weights:  81%|████████  | 357/443 [02:56<00:32,  2.62it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:56<00:32,  2.60it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:35,  2.42it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:44,  1.95it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 357/443 [02:56<00:44,  1.95it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:57<00:35,  2.41it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:57<00:35,  2.41it/s, Materializing param=model.layers.32.mlp.up_proj.weight]  
Loading weights:  81%|████████  | 358/443 [02:57<00:35,  2.41it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:57<00:35,  2.39it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:57<00:35,  2.39it/s, Materializing param=model.layers.32.mlp.up_proj.weight]  
Loading weights:  81%|████████  | 358/443 [02:57<00:35,  2.39it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:57<00:38,  2.22it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:57<00:38,  2.22it/s, Materializing param=model.layers.32.mlp.up_proj.weight]  
Loading weights:  81%|████████  | 358/443 [02:57<00:38,  2.22it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:57<00:40,  2.12it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:57<00:40,  2.12it/s, Materializing param=model.layers.32.mlp.up_proj.weight]  
Loading weights:  81%|████████  | 358/443 [02:57<00:40,  2.12it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:57<00:38,  2.19it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:57<00:38,  2.19it/s, Materializing param=model.layers.32.mlp.up_proj.weight]  
Loading weights:  81%|████████  | 358/443 [02:57<00:38,  2.19it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:38,  2.16it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:38,  2.16it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:38,  2.16it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████▏ | 360/443 [02:57<00:38,  2.16it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]        
Loading weights:  81%|████████  | 359/443 [02:57<00:36,  2.29it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:38,  2.16it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:40,  2.08it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████▏ | 360/443 [02:57<00:38,  2.16it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:36,  2.27it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:38,  2.16it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:40,  2.08it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:36,  2.29it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:38,  2.16it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:40,  2.08it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████▏ | 361/443 [02:57<00:37,  2.16it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:36,  2.29it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████▏ | 361/443 [02:57<00:37,  2.16it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:36,  2.27it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:36,  2.27it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████▏ | 360/443 [02:57<00:38,  2.16it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]        
Loading weights:  81%|████████▏ | 360/443 [02:57<00:39,  2.08it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]        
Loading weights:  81%|████████▏ | 360/443 [02:57<00:38,  2.16it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]
Loading weights:  81%|████████▏ | 360/443 [02:57<00:36,  2.29it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]        
Loading weights:  81%|████████▏ | 360/443 [02:57<00:39,  2.08it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]
Loading weights:  81%|████████▏ | 360/443 [02:57<00:36,  2.29it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]
Loading weights:  81%|████████▏ | 360/443 [02:57<00:36,  2.27it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]        
Loading weights:  81%|████████▏ | 361/443 [02:57<00:37,  2.16it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████▏ | 360/443 [02:57<00:36,  2.27it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]
Loading weights:  81%|████████▏ | 361/443 [02:57<00:39,  2.08it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████▏ | 361/443 [02:57<00:37,  2.16it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████▏ | 361/443 [02:57<00:39,  2.08it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████▏ | 361/443 [02:57<00:35,  2.29it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████▏ | 361/443 [02:57<00:35,  2.29it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████▏ | 361/443 [02:57<00:36,  2.27it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████▏ | 361/443 [02:57<00:36,  2.27it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:38,  2.20it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:38,  2.20it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████  | 359/443 [02:57<00:38,  2.20it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████▏ | 360/443 [02:57<00:37,  2.20it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]        
Loading weights:  81%|████████▏ | 360/443 [02:57<00:37,  2.20it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]
Loading weights:  81%|████████▏ | 361/443 [02:57<00:37,  2.20it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████▏ | 361/443 [02:57<00:37,  2.20it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:22,  3.55it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:21,  3.73it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:22,  3.55it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:22,  3.56it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:22,  3.55it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:21,  3.73it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:24,  3.34it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:22,  3.55it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:22,  3.56it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:21,  3.73it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:22,  3.56it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:24,  3.34it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:24,  3.34it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:22,  3.55it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:22,  3.55it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:24,  3.30it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:24,  3.30it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:24,  3.30it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:58<00:47,  1.80it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:58<00:47,  1.80it/s, Materializing param=model.layers.32.mlp.up_proj.weight]  
Loading weights:  81%|████████  | 358/443 [02:58<00:47,  1.80it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 359/443 [02:58<00:46,  1.80it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████  | 359/443 [02:58<00:46,  1.80it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████▏ | 360/443 [02:58<00:46,  1.80it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]        
Loading weights:  81%|████████▏ | 360/443 [02:58<00:46,  1.80it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]
Loading weights:  81%|████████▏ | 361/443 [02:58<00:45,  1.80it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████▏ | 361/443 [02:58<00:45,  1.80it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:58<00:44,  1.90it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
Loading weights:  81%|████████  | 358/443 [02:58<00:44,  1.90it/s, Materializing param=model.layers.32.mlp.up_proj.weight]  
Loading weights:  81%|████████  | 358/443 [02:58<00:44,  1.90it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
Loading weights:  81%|████████  | 359/443 [02:58<00:44,  1.90it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████  | 359/443 [02:58<00:44,  1.90it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
Loading weights:  81%|████████▏ | 360/443 [02:58<00:43,  1.90it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]        
Loading weights:  81%|████████▏ | 360/443 [02:58<00:43,  1.90it/s, Materializing param=model.layers.32.self_attn.k_norm.weight]
Loading weights:  81%|████████▏ | 361/443 [02:58<00:43,  1.90it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  81%|████████▏ | 361/443 [02:58<00:43,  1.90it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:45,  1.80it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:45,  1.80it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:42,  1.90it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 362/443 [02:58<00:42,  1.90it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:22,  3.55it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:22,  3.55it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:21,  3.73it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:22,  3.56it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:25,  3.20it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:25,  3.09it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:23,  3.34it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:22,  3.55it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:21,  3.73it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:22,  3.56it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:22,  3.55it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:25,  3.20it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:23,  3.34it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:25,  3.09it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:25,  3.20it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:25,  3.09it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:24,  3.30it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:22,  3.51it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 363/443 [02:58<00:24,  3.30it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:22,  3.51it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:22,  3.51it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:23,  3.32it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:23,  3.32it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:22,  3.50it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:23,  3.32it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:22,  3.50it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:22,  3.50it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:22,  3.50it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:22,  3.50it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:22,  3.50it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:24,  3.20it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:25,  3.09it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:24,  3.20it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:58<00:25,  3.09it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:59<00:28,  2.81it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:25,  3.01it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:59<00:28,  2.81it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:59<00:28,  2.81it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:25,  3.02it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:25,  3.01it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:24,  3.19it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:25,  3.01it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:23,  3.28it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:27,  2.81it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:24,  3.19it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:27,  2.81it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:24,  3.19it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:25,  3.01it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:26,  2.91it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:23,  3.28it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:25,  3.01it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:23,  3.28it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:26,  2.91it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:25,  3.02it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:25,  3.01it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:26,  2.91it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:25,  3.02it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:21,  3.59it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:21,  3.59it/s, Materializing param=model.layers.33.input_layernorm.weight] 
Loading weights:  83%|████████▎ | 366/443 [02:59<00:21,  3.59it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.33.input_layernorm.weight] 
Loading weights:  83%|████████▎ | 366/443 [02:59<00:21,  3.50it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:23,  3.34it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.33.input_layernorm.weight] 
Loading weights:  83%|████████▎ | 366/443 [02:59<00:21,  3.50it/s, Materializing param=model.layers.33.input_layernorm.weight] 
Loading weights:  83%|████████▎ | 366/443 [02:59<00:23,  3.34it/s, Materializing param=model.layers.33.input_layernorm.weight] 
Loading weights:  83%|████████▎ | 366/443 [02:59<00:21,  3.50it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:23,  3.34it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|████████▎ | 367/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.33.mlp.down_proj.weight]  
Loading weights:  83%|████████▎ | 366/443 [02:59<00:20,  3.71it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  83%|████████▎ | 367/443 [02:59<00:22,  3.34it/s, Materializing param=model.layers.33.mlp.down_proj.weight]  
Loading weights:  83%|████████▎ | 367/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  83%|████████▎ | 367/443 [02:59<00:22,  3.34it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:20,  3.71it/s, Materializing param=model.layers.33.input_layernorm.weight] 
Loading weights:  83%|████████▎ | 366/443 [02:59<00:20,  3.71it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  82%|████████▏ | 364/443 [02:59<00:30,  2.61it/s, Materializing param=model.layers.32.self_attn.q_norm.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.33.input_layernorm.weight] 
Loading weights:  83%|████████▎ | 366/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|████████▎ | 367/443 [02:59<00:20,  3.71it/s, Materializing param=model.layers.33.mlp.down_proj.weight]  
Loading weights:  82%|████████▏ | 364/443 [02:59<00:30,  2.61it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  82%|████████▏ | 364/443 [02:59<00:30,  2.61it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
Loading weights:  83%|████████▎ | 367/443 [02:59<00:20,  3.71it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 367/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.33.mlp.down_proj.weight]  
Loading weights:  83%|████████▎ | 367/443 [02:59<00:22,  3.45it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:29,  2.61it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  82%|████████▏ | 365/443 [02:59<00:29,  2.61it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
Loading weights:  83%|████████▎ | 366/443 [02:59<00:29,  2.61it/s, Materializing param=model.layers.33.input_layernorm.weight] 
Loading weights:  83%|████████▎ | 366/443 [02:59<00:29,  2.61it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|████████▎ | 367/443 [02:59<00:29,  2.61it/s, Materializing param=model.layers.33.mlp.down_proj.weight]  
Loading weights:  83%|████████▎ | 367/443 [02:59<00:29,  2.61it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:32,  2.28it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:41,  1.80it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:41,  1.79it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:41,  1.80it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 367/443 [03:01<00:44,  1.72it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:36,  2.03it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:41,  1.80it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:32,  2.28it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:41,  1.79it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:41,  1.80it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 367/443 [03:01<00:44,  1.72it/s, Materializing param=model.layers.33.mlp.down_proj.weight]  
Loading weights:  83%|████████▎ | 368/443 [03:01<00:41,  1.80it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:32,  2.28it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:41,  1.79it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 367/443 [03:01<00:44,  1.72it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:41,  1.80it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:36,  2.03it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 367/443 [03:01<00:51,  1.48it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:36,  2.03it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:43,  1.72it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 367/443 [03:01<00:51,  1.48it/s, Materializing param=model.layers.33.mlp.down_proj.weight]  
Loading weights:  83%|████████▎ | 369/443 [03:01<00:36,  2.03it/s, Materializing param=model.layers.33.mlp.up_proj.weight]  
Loading weights:  83%|████████▎ | 368/443 [03:01<00:43,  1.72it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:01<00:36,  2.03it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  83%|████████▎ | 367/443 [03:01<00:51,  1.48it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:50,  1.48it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:50,  1.48it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:01<00:32,  2.28it/s, Materializing param=model.layers.33.mlp.up_proj.weight]  
Loading weights:  83%|████████▎ | 369/443 [03:01<00:32,  2.28it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  83%|████████▎ | 367/443 [03:01<00:45,  1.68it/s, Materializing param=model.layers.33.input_layernorm.weight]
Loading weights:  83%|████████▎ | 367/443 [03:01<00:45,  1.68it/s, Materializing param=model.layers.33.mlp.down_proj.weight]  
Loading weights:  83%|████████▎ | 367/443 [03:01<00:45,  1.68it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:44,  1.68it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 368/443 [03:01<00:44,  1.68it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:42,  1.74it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:42,  1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight]  
Loading weights:  83%|████████▎ | 369/443 [03:02<00:42,  1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:42,  1.74it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:36,  2.01it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:42,  1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight]  
Loading weights:  83%|████████▎ | 369/443 [03:02<00:36,  2.01it/s, Materializing param=model.layers.33.mlp.up_proj.weight]  
Loading weights:  83%|████████▎ | 369/443 [03:02<00:42,  1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:36,  2.01it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:35,  2.06it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:35,  2.06it/s, Materializing param=model.layers.33.mlp.up_proj.weight]  
Loading weights:  83%|████████▎ | 369/443 [03:02<00:35,  2.06it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:42,  1.74it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:42,  1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight]  
Loading weights:  83%|████████▎ | 369/443 [03:02<00:42,  1.74it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:40,  1.85it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
Loading weights:  83%|████████▎ | 369/443 [03:02<00:40,  1.85it/s, Materializing param=model.layers.33.mlp.up_proj.weight]  
Loading weights:  83%|████████▎ | 369/443 [03:02<00:40,  1.85it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:32,  2.22it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:35,  2.03it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:32,  2.22it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:32,  2.22it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:35,  2.03it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:35,  2.03it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 371/443 [03:02<00:35,  2.03it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]        
Loading weights:  84%|████████▎ | 371/443 [03:02<00:35,  2.03it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]
Loading weights:  84%|████████▎ | 371/443 [03:02<00:32,  2.22it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]        
Loading weights:  84%|████████▎ | 371/443 [03:02<00:32,  2.22it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:34,  2.03it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:31,  2.22it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:34,  2.03it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:31,  2.22it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:21,  3.18it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:21,  3.18it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:21,  3.18it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:22,  3.16it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:22,  3.16it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:22,  3.16it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:40,  1.80it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:40,  1.81it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:40,  1.80it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:40,  1.80it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:40,  1.81it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:40,  1.81it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 371/443 [03:02<00:39,  1.80it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]        
Loading weights:  84%|████████▎ | 371/443 [03:02<00:39,  1.80it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]
Loading weights:  84%|████████▎ | 371/443 [03:02<00:39,  1.81it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]        
Loading weights:  84%|████████▎ | 371/443 [03:02<00:39,  1.81it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:39,  1.81it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:39,  1.81it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:39,  1.80it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:39,  1.80it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:40,  1.80it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:40,  1.80it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:40,  1.80it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 371/443 [03:02<00:39,  1.80it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]        
Loading weights:  84%|████████▎ | 371/443 [03:02<00:39,  1.80it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:39,  1.80it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:39,  1.80it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:38,  1.80it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:38,  1.80it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:38,  1.80it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:38,  1.80it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:38,  1.81it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:38,  1.81it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:36,  1.97it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:36,  1.97it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:36,  1.97it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:36,  2.01it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:36,  2.01it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:36,  2.01it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 371/443 [03:02<00:35,  2.01it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]        
Loading weights:  84%|████████▎ | 371/443 [03:02<00:35,  2.01it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:35,  2.01it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:35,  2.01it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:39,  1.85it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:39,  1.85it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 370/443 [03:02<00:39,  1.85it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
Loading weights:  84%|████████▎ | 371/443 [03:02<00:39,  1.85it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]        
Loading weights:  84%|████████▎ | 371/443 [03:02<00:39,  1.85it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:38,  1.85it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:38,  1.85it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▎ | 371/443 [03:02<00:36,  1.97it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]        
Loading weights:  84%|████████▎ | 371/443 [03:02<00:36,  1.97it/s, Materializing param=model.layers.33.self_attn.k_norm.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:35,  1.97it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 372/443 [03:02<00:35,  1.97it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:35,  1.97it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:35,  1.97it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:34,  2.01it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:34,  2.01it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:37,  1.85it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 373/443 [03:02<00:37,  1.85it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.03it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.03it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.02it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:21,  3.15it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.12it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:21,  3.18it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.03it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.03it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:21,  3.16it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.02it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.03it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.03it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:21,  3.18it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.02it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:21,  3.16it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:21,  3.15it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:22,  3.03it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:22,  3.03it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.13it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.12it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:21,  3.15it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.13it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.13it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.15it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.15it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:22,  3.12it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.12it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.14it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.14it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.14it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.12it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:21,  3.19it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:21,  3.19it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  84%|████████▍ | 374/443 [03:03<00:21,  3.19it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.19it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.19it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:21,  3.19it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:21,  3.19it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▌ | 377/443 [03:03<00:20,  3.19it/s, Materializing param=model.layers.34.input_layernorm.weight] 
Loading weights:  85%|████████▌ | 377/443 [03:03<00:20,  3.19it/s, Materializing param=model.layers.34.input_layernorm.weight]
Loading weights:  85%|████████▌ | 378/443 [03:03<00:20,  3.19it/s, Materializing param=model.layers.34.mlp.down_proj.weight]  
Loading weights:  85%|████████▌ | 378/443 [03:03<00:20,  3.19it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:17,  3.88it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:17,  3.88it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:17,  3.88it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.23it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:20,  3.31it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:20,  3.31it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.23it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:20,  3.31it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.23it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:20,  3.23it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:20,  3.23it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:20,  3.28it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:20,  3.28it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:20,  3.28it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:17,  3.74it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:17,  3.74it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:17,  3.74it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▌ | 377/443 [03:03<00:17,  3.88it/s, Materializing param=model.layers.34.input_layernorm.weight] 
Loading weights:  85%|████████▌ | 377/443 [03:03<00:17,  3.88it/s, Materializing param=model.layers.34.input_layernorm.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.15it/s, Materializing param=model.layers.33.self_attn.q_norm.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.15it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 375/443 [03:03<00:21,  3.15it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:21,  3.15it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:21,  3.15it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▌ | 377/443 [03:03<00:20,  3.15it/s, Materializing param=model.layers.34.input_layernorm.weight] 
Loading weights:  85%|████████▌ | 377/443 [03:03<00:20,  3.15it/s, Materializing param=model.layers.34.input_layernorm.weight]
Loading weights:  85%|████████▌ | 378/443 [03:03<00:20,  3.15it/s, Materializing param=model.layers.34.mlp.down_proj.weight]  
Loading weights:  85%|████████▌ | 378/443 [03:03<00:20,  3.15it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  85%|████████▌ | 377/443 [03:03<00:20,  3.23it/s, Materializing param=model.layers.34.input_layernorm.weight] 
Loading weights:  85%|████████▌ | 377/443 [03:03<00:19,  3.31it/s, Materializing param=model.layers.34.input_layernorm.weight] 
Loading weights:  85%|████████▌ | 377/443 [03:03<00:20,  3.23it/s, Materializing param=model.layers.34.input_layernorm.weight]
Loading weights:  85%|████████▌ | 377/443 [03:03<00:19,  3.31it/s, Materializing param=model.layers.34.input_layernorm.weight]
Loading weights:  85%|████████▌ | 378/443 [03:03<00:20,  3.23it/s, Materializing param=model.layers.34.mlp.down_proj.weight]  
Loading weights:  85%|████████▌ | 378/443 [03:03<00:20,  3.23it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  85%|████████▌ | 378/443 [03:03<00:19,  3.31it/s, Materializing param=model.layers.34.mlp.down_proj.weight]  
Loading weights:  85%|████████▌ | 378/443 [03:03<00:19,  3.31it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:17,  3.76it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:17,  3.76it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▍ | 376/443 [03:03<00:17,  3.76it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
Loading weights:  85%|████████▌ | 377/443 [03:03<00:17,  3.76it/s, Materializing param=model.layers.34.input_layernorm.weight] 
Loading weights:  85%|████████▌ | 377/443 [03:03<00:20,  3.28it/s, Materializing param=model.layers.34.input_layernorm.weight] 
Loading weights:  85%|████████▌ | 377/443 [03:03<00:17,  3.76it/s, Materializing param=model.layers.34.input_layernorm.weight]
Loading weights:  85%|████████▌ | 377/443 [03:03<00:20,  3.28it/s, Materializing param=model.layers.34.input_layernorm.weight]
Loading weights:  85%|████████▌ | 378/443 [03:03<00:19,  3.28it/s, Materializing param=model.layers.34.mlp.down_proj.weight]  
Loading weights:  85%|████████▌ | 378/443 [03:03<00:17,  3.76it/s, Materializing param=model.layers.34.mlp.down_proj.weight]  
Loading weights:  85%|████████▌ | 378/443 [03:03<00:19,  3.28it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  85%|████████▌ | 378/443 [03:03<00:17,  3.76it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  85%|████████▌ | 377/443 [03:03<00:17,  3.74it/s, Materializing param=model.layers.34.input_layernorm.weight] 
Loading weights:  85%|████████▌ | 377/443 [03:03<00:17,  3.74it/s, Materializing param=model.layers.34.input_layernorm.weight]
Loading weights:  85%|████████▌ | 378/443 [03:03<00:17,  3.74it/s, Materializing param=model.layers.34.mlp.down_proj.weight]  
Loading weights:  85%|████████▌ | 378/443 [03:03<00:17,  3.74it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:24,  2.61it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:21,  2.93it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:22,  2.82it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:24,  2.61it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:21,  2.93it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:24,  2.61it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:21,  2.93it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  85%|████████▌ | 378/443 [03:04<00:25,  2.51it/s, Materializing param=model.layers.34.input_layernorm.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:22,  2.82it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:22,  2.82it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  85%|████████▌ | 378/443 [03:04<00:25,  2.51it/s, Materializing param=model.layers.34.mlp.down_proj.weight]  
Loading weights:  86%|████████▌ | 379/443 [03:04<00:24,  2.64it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:22,  2.86it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  85%|████████▌ | 378/443 [03:04<00:25,  2.51it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:24,  2.64it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:20,  3.05it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:25,  2.51it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:24,  2.64it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:25,  2.51it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:20,  3.05it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:20,  3.05it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:22,  2.86it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:22,  2.86it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:21,  2.92it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:21,  2.92it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 379/443 [03:04<00:21,  2.92it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:04<00:21,  2.92it/s, Materializing param=model.layers.34.mlp.up_proj.weight]  
Loading weights:  86%|████████▌ | 380/443 [03:04<00:21,  2.92it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:22,  2.74it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:22,  2.74it/s, Materializing param=model.layers.34.mlp.up_proj.weight]  
Loading weights:  86%|████████▌ | 380/443 [03:05<00:22,  2.74it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:24,  2.52it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:24,  2.52it/s, Materializing param=model.layers.34.mlp.up_proj.weight]  
Loading weights:  86%|████████▌ | 380/443 [03:05<00:24,  2.52it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:24,  2.56it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:24,  2.56it/s, Materializing param=model.layers.34.mlp.up_proj.weight]  
Loading weights:  86%|████████▌ | 380/443 [03:05<00:24,  2.56it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:22,  2.74it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:22,  2.74it/s, Materializing param=model.layers.34.mlp.up_proj.weight]  
Loading weights:  86%|████████▌ | 380/443 [03:05<00:22,  2.74it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:24,  2.62it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:24,  2.62it/s, Materializing param=model.layers.34.mlp.up_proj.weight]  
Loading weights:  86%|████████▌ | 380/443 [03:05<00:24,  2.62it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:22,  2.78it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:05<00:22,  2.78it/s, Materializing param=model.layers.34.mlp.up_proj.weight]  
Loading weights:  86%|████████▌ | 380/443 [03:05<00:22,  2.78it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:33,  1.87it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:37,  1.67it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:31,  1.99it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:33,  1.87it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:37,  1.67it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:31,  1.99it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:33,  1.87it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:37,  1.67it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:31,  1.99it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:35,  1.75it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 382/443 [03:06<00:36,  1.67it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]        
Loading weights:  86%|████████▌ | 382/443 [03:06<00:32,  1.87it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]        
Loading weights:  86%|████████▌ | 382/443 [03:06<00:36,  1.67it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]
Loading weights:  86%|████████▌ | 382/443 [03:06<00:30,  1.99it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]        
Loading weights:  86%|████████▌ | 382/443 [03:06<00:32,  1.87it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:35,  1.75it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:34,  1.79it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:31,  1.99it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 382/443 [03:06<00:30,  1.99it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:35,  1.75it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:35,  1.67it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:35,  1.67it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:31,  1.99it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:30,  1.99it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:31,  1.99it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:32,  1.87it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:30,  1.99it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:32,  1.87it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▌ | 382/443 [03:06<00:30,  1.99it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]        
Loading weights:  86%|████████▌ | 382/443 [03:06<00:34,  1.75it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]        
Loading weights:  86%|████████▌ | 381/443 [03:06<00:34,  1.79it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 382/443 [03:06<00:30,  1.99it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]
Loading weights:  86%|████████▌ | 382/443 [03:06<00:34,  1.75it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:30,  1.99it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:34,  1.79it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:30,  1.99it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:34,  1.75it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:34,  1.75it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▌ | 382/443 [03:06<00:33,  1.79it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]        
Loading weights:  86%|████████▌ | 382/443 [03:06<00:33,  1.79it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:33,  1.79it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:33,  1.79it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:36,  1.70it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:36,  1.70it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:36,  1.70it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 382/443 [03:06<00:35,  1.70it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]        
Loading weights:  86%|████████▌ | 382/443 [03:06<00:35,  1.70it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:35,  1.70it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:35,  1.70it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:34,  1.70it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:34,  1.70it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:35,  1.67it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:35,  1.67it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:33,  1.75it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:33,  1.75it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:29,  1.99it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:29,  1.99it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:31,  1.87it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:31,  1.87it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:32,  1.79it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:29,  1.99it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:32,  1.79it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:29,  1.99it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:06<00:37,  1.67it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
Loading weights:  86%|████████▌ | 380/443 [03:06<00:37,  1.67it/s, Materializing param=model.layers.34.mlp.up_proj.weight]  
Loading weights:  86%|████████▌ | 380/443 [03:06<00:37,  1.67it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:37,  1.67it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 381/443 [03:06<00:37,  1.67it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
Loading weights:  86%|████████▌ | 382/443 [03:06<00:36,  1.67it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]        
Loading weights:  86%|████████▌ | 382/443 [03:06<00:36,  1.67it/s, Materializing param=model.layers.34.self_attn.k_norm.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:35,  1.67it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  86%|████████▋ | 383/443 [03:06<00:35,  1.67it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:35,  1.67it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 384/443 [03:06<00:35,  1.67it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  2.98it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:20,  2.87it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  3.05it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:20,  2.87it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  3.05it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:20,  2.87it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  2.94it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  3.05it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  2.94it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  2.98it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:18,  3.08it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  2.94it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  2.98it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:18,  3.08it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:18,  3.08it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:18,  3.08it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:18,  3.08it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:20,  2.87it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:20,  2.87it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:20,  2.87it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:19,  2.87it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:19,  2.87it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:19,  2.87it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:20,  2.81it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:19,  2.87it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:20,  2.81it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:20,  2.81it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:20,  2.81it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:20,  2.81it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:19,  2.81it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:19,  2.81it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  2.97it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  2.97it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 385/443 [03:07<00:19,  2.97it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:19,  2.97it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:19,  2.97it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:18,  2.97it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:18,  2.97it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:17,  3.32it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:17,  3.32it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:17,  3.32it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:16,  3.32it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:16,  3.32it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:17,  3.19it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:17,  3.19it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:17,  3.19it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:17,  3.19it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:17,  3.19it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:14,  3.74it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:14,  3.74it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:14,  3.74it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:17,  3.26it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:18,  3.15it/s, Materializing param=model.layers.34.self_attn.q_norm.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:18,  3.15it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:18,  3.15it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:17,  3.26it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:17,  3.15it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:17,  3.15it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 386/443 [03:07<00:17,  3.26it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:17,  3.26it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  87%|████████▋ | 387/443 [03:07<00:17,  3.26it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:16,  3.32it/s, Materializing param=model.layers.35.input_layernorm.weight] 
Loading weights:  88%|████████▊ | 388/443 [03:07<00:17,  3.19it/s, Materializing param=model.layers.35.input_layernorm.weight] 
Loading weights:  88%|████████▊ | 388/443 [03:07<00:17,  3.19it/s, Materializing param=model.layers.35.input_layernorm.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:14,  3.74it/s, Materializing param=model.layers.35.input_layernorm.weight] 
Loading weights:  88%|████████▊ | 388/443 [03:07<00:13,  4.18it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:13,  4.12it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:12,  4.28it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:13,  4.18it/s, Materializing param=model.layers.35.input_layernorm.weight] 
Loading weights:  88%|████████▊ | 388/443 [03:07<00:13,  4.18it/s, Materializing param=model.layers.35.input_layernorm.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:12,  4.28it/s, Materializing param=model.layers.35.input_layernorm.weight] 
Loading weights:  88%|████████▊ | 388/443 [03:07<00:13,  4.12it/s, Materializing param=model.layers.35.input_layernorm.weight] 
Loading weights:  88%|████████▊ | 388/443 [03:07<00:12,  4.28it/s, Materializing param=model.layers.35.input_layernorm.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:13,  4.12it/s, Materializing param=model.layers.35.input_layernorm.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:14,  3.74it/s, Materializing param=model.layers.35.input_layernorm.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:16,  3.32it/s, Materializing param=model.layers.35.input_layernorm.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:16,  3.26it/s, Materializing param=model.layers.35.input_layernorm.weight] 
Loading weights:  88%|████████▊ | 389/443 [03:07<00:14,  3.74it/s, Materializing param=model.layers.35.mlp.down_proj.weight]  
Loading weights:  88%|████████▊ | 389/443 [03:07<00:16,  3.19it/s, Materializing param=model.layers.35.mlp.down_proj.weight]  
Loading weights:  88%|████████▊ | 389/443 [03:07<00:13,  4.12it/s, Materializing param=model.layers.35.mlp.down_proj.weight]  
Loading weights:  88%|████████▊ | 389/443 [03:07<00:14,  3.74it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 389/443 [03:07<00:16,  3.19it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 389/443 [03:07<00:12,  4.28it/s, Materializing param=model.layers.35.mlp.down_proj.weight]  
Loading weights:  88%|████████▊ | 389/443 [03:07<00:13,  4.12it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:17,  3.15it/s, Materializing param=model.layers.35.input_layernorm.weight] 
Loading weights:  88%|████████▊ | 389/443 [03:07<00:16,  3.32it/s, Materializing param=model.layers.35.mlp.down_proj.weight]  
Loading weights:  88%|████████▊ | 389/443 [03:07<00:12,  4.28it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:16,  3.26it/s, Materializing param=model.layers.35.input_layernorm.weight]
Loading weights:  88%|████████▊ | 389/443 [03:07<00:16,  3.32it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 388/443 [03:07<00:17,  3.15it/s, Materializing param=model.layers.35.input_layernorm.weight]
Loading weights:  88%|████████▊ | 389/443 [03:07<00:12,  4.18it/s, Materializing param=model.layers.35.mlp.down_proj.weight]  
Loading weights:  88%|████████▊ | 389/443 [03:07<00:12,  4.18it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 389/443 [03:07<00:17,  3.15it/s, Materializing param=model.layers.35.mlp.down_proj.weight]  
Loading weights:  88%|████████▊ | 389/443 [03:07<00:17,  3.15it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 389/443 [03:07<00:16,  3.26it/s, Materializing param=model.layers.35.mlp.down_proj.weight]  
Loading weights:  88%|████████▊ | 389/443 [03:07<00:16,  3.26it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.46it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.51it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.43it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.48it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:22,  2.32it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:23,  2.29it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.46it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:23,  2.26it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.51it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.43it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.48it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:22,  2.32it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:23,  2.29it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.46it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.51it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.43it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:23,  2.26it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:22,  2.32it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.48it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:23,  2.29it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.46it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:23,  2.26it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:09<00:20,  2.51it/s, Materializing param=model.layers.35.mlp.up_proj.weight]  
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.46it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:09<00:20,  2.51it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 390/443 [03:09<00:21,  2.46it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:09<00:23,  2.22it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:09<00:23,  2.22it/s, Materializing param=model.layers.35.mlp.up_proj.weight]  
Loading weights:  88%|████████▊ | 391/443 [03:09<00:23,  2.22it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:22,  2.26it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:22,  2.26it/s, Materializing param=model.layers.35.mlp.up_proj.weight]  
Loading weights:  88%|████████▊ | 391/443 [03:10<00:22,  2.26it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:22,  2.35it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:22,  2.35it/s, Materializing param=model.layers.35.mlp.up_proj.weight]  
Loading weights:  88%|████████▊ | 391/443 [03:10<00:22,  2.35it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:24,  2.13it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:24,  2.13it/s, Materializing param=model.layers.35.mlp.up_proj.weight]  
Loading weights:  88%|████████▊ | 391/443 [03:10<00:24,  2.13it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:26,  2.00it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:26,  2.00it/s, Materializing param=model.layers.35.mlp.up_proj.weight]  
Loading weights:  88%|████████▊ | 391/443 [03:10<00:26,  2.00it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:24,  2.16it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:24,  2.16it/s, Materializing param=model.layers.35.mlp.up_proj.weight]  
Loading weights:  88%|████████▊ | 391/443 [03:10<00:24,  2.16it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:24,  2.10it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
Loading weights:  88%|████████▊ | 391/443 [03:10<00:24,  2.10it/s, Materializing param=model.layers.35.mlp.up_proj.weight]  
Loading weights:  88%|████████▊ | 391/443 [03:10<00:24,  2.10it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:24,  2.04it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.19it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:24,  2.09it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:24,  2.09it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:24,  2.04it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:24,  2.09it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:24,  2.04it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:22,  2.26it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.19it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  89%|████████▊ | 393/443 [03:10<00:24,  2.04it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]        
Loading weights:  89%|████████▊ | 393/443 [03:10<00:23,  2.09it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]        
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.19it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  89%|████████▊ | 393/443 [03:10<00:24,  2.04it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]
Loading weights:  89%|████████▊ | 393/443 [03:10<00:23,  2.09it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:22,  2.26it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:23,  2.04it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▊ | 393/443 [03:10<00:22,  2.19it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]        
Loading weights:  88%|████████▊ | 392/443 [03:10<00:22,  2.26it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:23,  2.09it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:23,  2.04it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▊ | 393/443 [03:10<00:22,  2.19it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:23,  2.09it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▊ | 393/443 [03:10<00:22,  2.26it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]        
Loading weights:  89%|████████▉ | 394/443 [03:10<00:22,  2.19it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▊ | 393/443 [03:10<00:22,  2.26it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:22,  2.19it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:21,  2.26it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:21,  2.26it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.18it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.18it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.18it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  89%|████████▊ | 393/443 [03:10<00:22,  2.18it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]        
Loading weights:  89%|████████▊ | 393/443 [03:10<00:22,  2.18it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:22,  2.18it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:22,  2.18it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:14,  3.30it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:13,  3.44it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:13,  3.48it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:14,  3.30it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:13,  3.44it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:13,  3.53it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:14,  3.30it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:13,  3.48it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:13,  3.44it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:13,  3.53it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:13,  3.48it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:13,  3.53it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.16it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.20it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.16it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:24,  2.06it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.20it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:24,  2.06it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.16it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:23,  2.20it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  88%|████████▊ | 392/443 [03:10<00:24,  2.06it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
Loading weights:  89%|████████▊ | 393/443 [03:10<00:23,  2.16it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]        
Loading weights:  89%|████████▊ | 393/443 [03:10<00:24,  2.06it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]        
Loading weights:  89%|████████▊ | 393/443 [03:10<00:23,  2.16it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]
Loading weights:  89%|████████▊ | 393/443 [03:10<00:24,  2.06it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]
Loading weights:  89%|████████▊ | 393/443 [03:10<00:22,  2.20it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]        
Loading weights:  89%|████████▊ | 393/443 [03:10<00:22,  2.20it/s, Materializing param=model.layers.35.self_attn.k_norm.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:22,  2.16it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:23,  2.06it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:23,  2.06it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:22,  2.16it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:22,  2.20it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 394/443 [03:10<00:22,  2.20it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:22,  2.18it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:22,  2.18it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:22,  2.16it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:22,  2.16it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:21,  2.20it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:23,  2.06it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:21,  2.20it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 395/443 [03:10<00:23,  2.06it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:14,  3.30it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.44it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.53it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:14,  3.30it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.53it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.44it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:12,  3.68it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:12,  3.61it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:13,  3.46it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:12,  3.61it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:12,  3.68it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:12,  3.61it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:13,  3.46it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:12,  3.68it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:13,  3.46it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:14,  3.29it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:14,  3.29it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:14,  3.29it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:13,  3.29it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:13,  3.29it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:13,  3.29it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:13,  3.29it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.48it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.48it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:12,  3.58it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:12,  3.58it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:12,  3.58it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:12,  3.58it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:12,  3.58it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.36it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.36it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  89%|████████▉ | 396/443 [03:11<00:13,  3.36it/s, Materializing param=model.layers.35.self_attn.q_norm.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:13,  3.36it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 397/443 [03:11<00:13,  3.36it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:13,  3.36it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:13,  3.36it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:12,  3.61it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:12,  3.68it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:12,  3.61it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:13,  3.39it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:12,  3.68it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:11,  3.82it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:11,  3.82it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:11,  3.82it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:09,  4.70it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:09,  4.74it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:12,  3.58it/s, Materializing param=model.layers.36.input_layernorm.weight] 
Loading weights:  90%|█████████ | 399/443 [03:11<00:09,  4.79it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:09,  4.70it/s, Materializing param=model.layers.36.input_layernorm.weight] 
Loading weights:  90%|█████████ | 399/443 [03:11<00:13,  3.29it/s, Materializing param=model.layers.36.input_layernorm.weight] 
Loading weights:  90%|█████████ | 399/443 [03:11<00:09,  4.74it/s, Materializing param=model.layers.36.input_layernorm.weight] 
Loading weights:  90%|█████████ | 399/443 [03:11<00:12,  3.58it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:09,  4.70it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:09,  4.74it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:09,  4.79it/s, Materializing param=model.layers.36.input_layernorm.weight] 
Loading weights:  90%|█████████ | 399/443 [03:11<00:13,  3.29it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:09,  4.79it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:11,  3.82it/s, Materializing param=model.layers.36.input_layernorm.weight] 
Loading weights:  90%|█████████ | 399/443 [03:11<00:11,  3.82it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  90%|█████████ | 400/443 [03:11<00:13,  3.29it/s, Materializing param=model.layers.36.mlp.down_proj.weight]  
Loading weights:  90%|█████████ | 400/443 [03:11<00:08,  4.79it/s, Materializing param=model.layers.36.mlp.down_proj.weight]  
Loading weights:  90%|█████████ | 400/443 [03:11<00:08,  4.79it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  90%|█████████ | 400/443 [03:11<00:13,  3.29it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:13,  3.36it/s, Materializing param=model.layers.36.input_layernorm.weight] 
Loading weights:  90%|█████████ | 400/443 [03:11<00:09,  4.70it/s, Materializing param=model.layers.36.mlp.down_proj.weight]  
Loading weights:  90%|█████████ | 400/443 [03:11<00:11,  3.82it/s, Materializing param=model.layers.36.mlp.down_proj.weight]  
Loading weights:  90%|█████████ | 400/443 [03:11<00:09,  4.70it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  90%|█████████ | 400/443 [03:11<00:11,  3.82it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:13,  3.36it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  90%|█████████ | 400/443 [03:11<00:12,  3.36it/s, Materializing param=model.layers.36.mlp.down_proj.weight]  
Loading weights:  90%|█████████ | 400/443 [03:11<00:12,  3.36it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:10,  4.20it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:10,  4.20it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|████████▉ | 398/443 [03:11<00:10,  4.20it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
Loading weights:  90%|█████████ | 399/443 [03:11<00:10,  4.20it/s, Materializing param=model.layers.36.input_layernorm.weight] 
Loading weights:  90%|█████████ | 399/443 [03:11<00:10,  4.20it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:14,  2.89it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:16,  2.55it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:18,  2.30it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:14,  2.85it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:17,  2.43it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:16,  2.55it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:14,  2.85it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  90%|█████████ | 400/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:18,  2.30it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  90%|█████████ | 400/443 [03:13<00:09,  4.74it/s, Materializing param=model.layers.36.mlp.down_proj.weight]  
Loading weights:  91%|█████████ | 401/443 [03:13<00:17,  2.43it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:16,  2.55it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:14,  2.85it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:18,  2.30it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  90%|█████████ | 400/443 [03:13<00:09,  4.74it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  90%|█████████ | 400/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.mlp.down_proj.weight]  
Loading weights:  91%|█████████ | 401/443 [03:13<00:17,  2.43it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:14,  2.89it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  90%|█████████ | 400/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  90%|█████████ | 400/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.input_layernorm.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:14,  2.89it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  90%|█████████ | 400/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.mlp.down_proj.weight]  
Loading weights:  90%|█████████ | 400/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 401/443 [03:13<00:18,  2.33it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:24,  1.65it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:21,  1.89it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:22,  1.79it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:27,  1.49it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:21,  1.92it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:23,  1.77it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:25,  1.58it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:27,  1.47it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:21,  1.89it/s, Materializing param=model.layers.36.mlp.up_proj.weight]  
Loading weights:  91%|█████████ | 402/443 [03:15<00:22,  1.79it/s, Materializing param=model.layers.36.mlp.up_proj.weight]  
Loading weights:  91%|█████████ | 402/443 [03:15<00:25,  1.58it/s, Materializing param=model.layers.36.mlp.up_proj.weight]  
Loading weights:  91%|█████████ | 402/443 [03:15<00:21,  1.89it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:23,  1.77it/s, Materializing param=model.layers.36.mlp.up_proj.weight]  
Loading weights:  91%|█████████ | 402/443 [03:15<00:27,  1.47it/s, Materializing param=model.layers.36.mlp.up_proj.weight]  
Loading weights:  91%|█████████ | 402/443 [03:15<00:22,  1.79it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:27,  1.49it/s, Materializing param=model.layers.36.mlp.up_proj.weight]  
Loading weights:  91%|█████████ | 402/443 [03:15<00:25,  1.58it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:23,  1.77it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:27,  1.47it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:27,  1.49it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:21,  1.92it/s, Materializing param=model.layers.36.mlp.up_proj.weight]  
Loading weights:  91%|█████████ | 402/443 [03:15<00:24,  1.65it/s, Materializing param=model.layers.36.mlp.up_proj.weight]  
Loading weights:  91%|█████████ | 402/443 [03:15<00:24,  1.65it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 402/443 [03:15<00:21,  1.92it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:36,  1.09it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:39,  1.02it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:30,  1.30it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:39,  1.02it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:36,  1.09it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:30,  1.30it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:39,  1.02it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:36,  1.09it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:30,  1.30it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:33,  1.20it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:33,  1.20it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:33,  1.20it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 404/443 [03:17<00:30,  1.30it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]        
Loading weights:  91%|█████████ | 404/443 [03:17<00:38,  1.02it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]        
Loading weights:  91%|█████████ | 404/443 [03:17<00:38,  1.02it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]
Loading weights:  91%|█████████ | 404/443 [03:17<00:30,  1.30it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:30,  1.31it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 404/443 [03:17<00:32,  1.20it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]        
Loading weights:  91%|█████████ | 404/443 [03:17<00:32,  1.20it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:38,  1.03it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:35,  1.13it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:37,  1.02it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:38,  1.03it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:32,  1.22it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:31,  1.20it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:37,  1.02it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:35,  1.13it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:38,  1.03it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:31,  1.20it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:35,  1.13it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:32,  1.22it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:30,  1.31it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:32,  1.22it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████ | 404/443 [03:17<00:37,  1.03it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]        
Loading weights:  91%|█████████ | 404/443 [03:17<00:34,  1.13it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]        
Loading weights:  91%|█████████ | 404/443 [03:17<00:32,  1.22it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]        
Loading weights:  91%|█████████ | 404/443 [03:17<00:37,  1.03it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]
Loading weights:  91%|█████████ | 404/443 [03:17<00:34,  1.13it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]
Loading weights:  91%|█████████ | 404/443 [03:17<00:32,  1.22it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:36,  1.03it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████ | 403/443 [03:17<00:30,  1.31it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:36,  1.03it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:31,  1.22it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:33,  1.13it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:31,  1.22it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:33,  1.13it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████ | 404/443 [03:17<00:29,  1.31it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]        
Loading weights:  91%|█████████ | 404/443 [03:17<00:29,  1.31it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:28,  1.31it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:28,  1.31it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:21,  1.78it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]
Loading weights:  91%|█████████ | 404/443 [03:17<00:29,  1.33it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:21,  1.78it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████ | 404/443 [03:17<00:29,  1.33it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]        
Loading weights:  92%|█████████▏| 406/443 [03:17<00:18,  2.05it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████ | 404/443 [03:17<00:29,  1.33it/s, Materializing param=model.layers.36.self_attn.k_norm.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:20,  1.83it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:18,  1.98it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:19,  1.91it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:18,  1.99it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:21,  1.78it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:18,  1.98it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:20,  1.83it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:19,  1.91it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:18,  1.99it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:28,  1.33it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:20,  1.83it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:19,  1.91it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:18,  1.99it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:18,  1.98it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  91%|█████████▏| 405/443 [03:17<00:28,  1.33it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:20,  1.78it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:18,  2.05it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:20,  1.78it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:27,  1.33it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:27,  1.33it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:18,  2.05it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:20,  1.83it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:20,  1.83it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 406/443 [03:17<00:20,  1.83it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:18,  1.95it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:18,  1.99it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:18,  1.99it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:18,  1.99it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:18,  1.95it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:19,  1.89it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:18,  1.91it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:19,  1.83it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:19,  1.89it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:19,  1.88it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:19,  1.83it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:18,  1.95it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:19,  1.89it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:17,  2.03it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:19,  1.88it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:18,  1.91it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:19,  1.88it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:17,  2.03it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:17,  1.99it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:17,  2.03it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:17,  1.99it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:16,  2.14it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:18,  1.88it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:17,  2.03it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:16,  2.14it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:18,  1.88it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:17,  2.03it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:16,  2.14it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:16,  2.08it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:16,  2.08it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:16,  2.08it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:18,  1.89it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:18,  1.89it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:19,  1.83it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 407/443 [03:18<00:19,  1.83it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:16,  2.08it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:16,  2.08it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:16,  2.08it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:16,  2.08it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:16,  2.08it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  93%|█████████▎| 410/443 [03:18<00:15,  2.08it/s, Materializing param=model.layers.37.input_layernorm.weight] 
Loading weights:  93%|█████████▎| 410/443 [03:18<00:15,  2.08it/s, Materializing param=model.layers.37.input_layernorm.weight]
Loading weights:  93%|█████████▎| 411/443 [03:18<00:15,  2.08it/s, Materializing param=model.layers.37.mlp.down_proj.weight]  
Loading weights:  93%|█████████▎| 411/443 [03:18<00:15,  2.08it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:15,  2.19it/s, Materializing param=model.layers.36.self_attn.q_norm.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:12,  2.65it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:12,  2.65it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:12,  2.65it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:15,  2.19it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 408/443 [03:18<00:15,  2.19it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:15,  2.19it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:15,  2.19it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:14,  2.38it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:14,  2.38it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:14,  2.38it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:13,  2.53it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:14,  2.31it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:13,  2.53it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:14,  2.31it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:13,  2.53it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:14,  2.31it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  93%|█████████▎| 410/443 [03:18<00:13,  2.38it/s, Materializing param=model.layers.37.input_layernorm.weight] 
Loading weights:  93%|█████████▎| 410/443 [03:18<00:13,  2.38it/s, Materializing param=model.layers.37.input_layernorm.weight]
Loading weights:  93%|█████████▎| 411/443 [03:18<00:13,  2.38it/s, Materializing param=model.layers.37.mlp.down_proj.weight]  
Loading weights:  93%|█████████▎| 411/443 [03:18<00:13,  2.38it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:13,  2.52it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:12,  2.63it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:13,  2.52it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:12,  2.63it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:13,  2.52it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  92%|█████████▏| 409/443 [03:18<00:12,  2.63it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
Loading weights:  93%|█████████▎| 410/443 [03:18<00:15,  2.19it/s, Materializing param=model.layers.37.input_layernorm.weight] 
Loading weights:  93%|█████████▎| 410/443 [03:18<00:12,  2.65it/s, Materializing param=model.layers.37.input_layernorm.weight] 
Loading weights:  93%|█████████▎| 410/443 [03:18<00:12,  2.65it/s, Materializing param=model.layers.37.input_layernorm.weight]
Loading weights:  93%|█████████▎| 410/443 [03:18<00:15,  2.19it/s, Materializing param=model.layers.37.input_layernorm.weight]
Loading weights:  93%|█████████▎| 411/443 [03:18<00:14,  2.19it/s, Materializing param=model.layers.37.mlp.down_proj.weight]  
Loading weights:  93%|█████████▎| 411/443 [03:18<00:12,  2.65it/s, Materializing param=model.layers.37.mlp.down_proj.weight]  
Loading weights:  93%|█████████▎| 411/443 [03:18<00:12,  2.65it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 411/443 [03:18<00:14,  2.19it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 410/443 [03:18<00:14,  2.31it/s, Materializing param=model.layers.37.input_layernorm.weight] 
Loading weights:  93%|█████████▎| 410/443 [03:18<00:14,  2.31it/s, Materializing param=model.layers.37.input_layernorm.weight]
Loading weights:  93%|█████████▎| 411/443 [03:18<00:13,  2.31it/s, Materializing param=model.layers.37.mlp.down_proj.weight]  
Loading weights:  93%|█████████▎| 411/443 [03:18<00:13,  2.31it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 410/443 [03:18<00:13,  2.52it/s, Materializing param=model.layers.37.input_layernorm.weight] 
Loading weights:  93%|█████████▎| 410/443 [03:18<00:13,  2.52it/s, Materializing param=model.layers.37.input_layernorm.weight]
Loading weights:  93%|█████████▎| 410/443 [03:18<00:13,  2.53it/s, Materializing param=model.layers.37.input_layernorm.weight] 
Loading weights:  93%|█████████▎| 410/443 [03:18<00:13,  2.53it/s, Materializing param=model.layers.37.input_layernorm.weight]
Loading weights:  93%|█████████▎| 411/443 [03:18<00:12,  2.53it/s, Materializing param=model.layers.37.mlp.down_proj.weight]  
Loading weights:  93%|█████████▎| 411/443 [03:18<00:12,  2.53it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 410/443 [03:18<00:12,  2.63it/s, Materializing param=model.layers.37.input_layernorm.weight] 
Loading weights:  93%|█████████▎| 410/443 [03:18<00:12,  2.63it/s, Materializing param=model.layers.37.input_layernorm.weight]
Loading weights:  93%|█████████▎| 411/443 [03:18<00:12,  2.63it/s, Materializing param=model.layers.37.mlp.down_proj.weight]  
Loading weights:  93%|█████████▎| 411/443 [03:18<00:12,  2.63it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 411/443 [03:19<00:17,  1.88it/s, Materializing param=model.layers.37.input_layernorm.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.21it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 411/443 [03:19<00:17,  1.88it/s, Materializing param=model.layers.37.mlp.down_proj.weight]  
Loading weights:  93%|█████████▎| 411/443 [03:19<00:17,  1.88it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.08it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:13,  2.23it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:15,  2.05it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.10it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:13,  2.28it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.17it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.21it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.21it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:13,  2.23it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:15,  2.05it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.08it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.17it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.10it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:15,  2.05it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.10it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:13,  2.23it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.17it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:14,  2.08it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:16,  1.88it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:16,  1.88it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:13,  2.28it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 412/443 [03:19<00:13,  2.28it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:22,  1.34it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:22,  1.35it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.44it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.47it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:21,  1.43it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:21,  1.40it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.47it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.44it/s, Materializing param=model.layers.37.mlp.up_proj.weight]  
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.44it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:21,  1.40it/s, Materializing param=model.layers.37.mlp.up_proj.weight]  
Loading weights:  93%|█████████▎| 413/443 [03:21<00:22,  1.34it/s, Materializing param=model.layers.37.mlp.up_proj.weight]  
Loading weights:  93%|█████████▎| 413/443 [03:21<00:21,  1.43it/s, Materializing param=model.layers.37.mlp.up_proj.weight]  
Loading weights:  93%|█████████▎| 413/443 [03:21<00:22,  1.35it/s, Materializing param=model.layers.37.mlp.up_proj.weight]  
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.47it/s, Materializing param=model.layers.37.mlp.up_proj.weight]  
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.44it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:22,  1.34it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:21,  1.43it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:21,  1.40it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.47it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:22,  1.35it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.44it/s, Materializing param=model.layers.37.mlp.up_proj.weight]  
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.44it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:21<00:20,  1.43it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  93%|█████████▎| 414/443 [03:21<00:20,  1.43it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  94%|█████████▎| 415/443 [03:21<00:19,  1.43it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]        
Loading weights:  94%|█████████▎| 415/443 [03:21<00:19,  1.43it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.47it/s, Materializing param=model.layers.37.mlp.up_proj.weight]  
Loading weights:  94%|█████████▍| 416/443 [03:21<00:18,  1.43it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 416/443 [03:21<00:18,  1.43it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  93%|█████████▎| 413/443 [03:21<00:20,  1.47it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:09,  2.62it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:09,  2.62it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:09,  2.62it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:19,  1.49it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:19,  1.49it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:19,  1.49it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  94%|█████████▎| 415/443 [03:22<00:18,  1.49it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]        
Loading weights:  94%|█████████▎| 415/443 [03:22<00:18,  1.49it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:18,  1.49it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:18,  1.49it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:19,  1.50it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:19,  1.50it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:19,  1.50it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  94%|█████████▎| 415/443 [03:22<00:18,  1.50it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]        
Loading weights:  94%|█████████▎| 415/443 [03:22<00:18,  1.50it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:18,  1.50it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:18,  1.50it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:17,  1.49it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:17,  1.49it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:17,  1.50it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:17,  1.50it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:20,  1.45it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:20,  1.45it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:20,  1.45it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  94%|█████████▎| 415/443 [03:22<00:19,  1.45it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]        
Loading weights:  94%|█████████▎| 415/443 [03:22<00:19,  1.45it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:18,  1.45it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:18,  1.45it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:17,  1.45it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:17,  1.45it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.62it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.62it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.85it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.85it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.85it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:21,  1.36it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:21,  1.36it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:21,  1.36it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  94%|█████████▎| 415/443 [03:22<00:20,  1.36it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]        
Loading weights:  94%|█████████▎| 415/443 [03:22<00:20,  1.36it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:19,  1.45it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:19,  1.45it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:19,  1.45it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  94%|█████████▎| 415/443 [03:22<00:19,  1.45it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]        
Loading weights:  94%|█████████▎| 415/443 [03:22<00:19,  1.45it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.76it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.76it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.76it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.76it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.76it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:08,  2.87it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:08,  2.87it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:08,  2.87it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.87it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.87it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:21,  1.34it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:21,  1.34it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:21,  1.34it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  94%|█████████▎| 415/443 [03:22<00:20,  1.34it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]        
Loading weights:  94%|█████████▎| 415/443 [03:22<00:20,  1.34it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:20,  1.34it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:20,  1.34it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:18,  1.45it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:18,  1.45it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:17,  1.45it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:17,  1.45it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:20,  1.44it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:20,  1.44it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  93%|█████████▎| 414/443 [03:22<00:20,  1.44it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
Loading weights:  94%|█████████▎| 415/443 [03:22<00:19,  1.44it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]        
Loading weights:  94%|█████████▎| 415/443 [03:22<00:19,  1.44it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:19,  1.36it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:19,  1.36it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:19,  1.36it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:19,  1.36it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:08,  2.81it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:08,  2.81it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:08,  2.81it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:19,  1.34it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:19,  1.34it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:13,  2.07it/s, Materializing param=model.layers.37.self_attn.k_norm.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:13,  2.07it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 416/443 [03:22<00:13,  2.07it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.68it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.68it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.68it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.68it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.68it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.68it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.68it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.77it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.68it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.77it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:09,  2.77it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.68it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.77it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.68it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:08,  2.77it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:12,  2.07it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 417/443 [03:22<00:12,  2.07it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:12,  2.07it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  94%|█████████▍| 418/443 [03:22<00:12,  2.07it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:11,  2.07it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 419/443 [03:22<00:11,  2.07it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:07,  3.19it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.32it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.75it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:08,  2.76it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.32it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:07,  3.19it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:08,  2.76it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.32it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:07,  3.19it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:08,  2.76it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.75it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 419/443 [03:23<00:08,  2.88it/s, Materializing param=model.layers.37.self_attn.q_norm.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.75it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 419/443 [03:23<00:08,  2.88it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 419/443 [03:23<00:08,  2.88it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:07,  2.88it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:07,  2.88it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.48it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.48it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▍| 420/443 [03:23<00:06,  3.48it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
Loading weights:  95%|█████████▌| 421/443 [03:23<00:06,  3.48it/s, Materializing param=model.layers.38.input_layernorm.weight] 
Loading weights:  95%|█████████▌| 421/443 [03:23<00:06,  3.48it/s, Materializing param=model.layers.38.input_layernorm.weight]
Loading weights:  95%|█████████▌| 421/443 [03:23<00:06,  3.32it/s, Materializing param=model.layers.38.input_layernorm.weight] 
Loading weights:  95%|█████████▌| 421/443 [03:23<00:06,  3.32it/s, Materializing param=model.layers.38.input_layernorm.weight]
Loading weights:  95%|█████████▌| 422/443 [03:23<00:06,  3.32it/s, Materializing param=model.layers.38.mlp.down_proj.weight]  
Loading weights:  95%|█████████▌| 422/443 [03:23<00:06,  3.32it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 421/443 [03:23<00:06,  3.19it/s, Materializing param=model.layers.38.input_layernorm.weight] 
Loading weights:  95%|█████████▌| 421/443 [03:23<00:06,  3.19it/s, Materializing param=model.layers.38.input_layernorm.weight]
Loading weights:  95%|█████████▌| 421/443 [03:23<00:07,  2.76it/s, Materializing param=model.layers.38.input_layernorm.weight] 
Loading weights:  95%|█████████▌| 421/443 [03:23<00:07,  2.76it/s, Materializing param=model.layers.38.input_layernorm.weight]
Loading weights:  95%|█████████▌| 422/443 [03:23<00:06,  3.19it/s, Materializing param=model.layers.38.mlp.down_proj.weight]  
Loading weights:  95%|█████████▌| 422/443 [03:23<00:07,  2.76it/s, Materializing param=model.layers.38.mlp.down_proj.weight]  
Loading weights:  95%|█████████▌| 422/443 [03:23<00:06,  3.19it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 421/443 [03:23<00:07,  2.88it/s, Materializing param=model.layers.38.input_layernorm.weight] 
Loading weights:  95%|█████████▌| 422/443 [03:23<00:07,  2.76it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 421/443 [03:23<00:05,  3.75it/s, Materializing param=model.layers.38.input_layernorm.weight] 
Loading weights:  95%|█████████▌| 421/443 [03:23<00:07,  2.88it/s, Materializing param=model.layers.38.input_layernorm.weight]
Loading weights:  95%|█████████▌| 421/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.38.input_layernorm.weight] 
Loading weights:  95%|█████████▌| 421/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.38.input_layernorm.weight]
Loading weights:  95%|█████████▌| 422/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.38.mlp.down_proj.weight]  
Loading weights:  95%|█████████▌| 422/443 [03:23<00:07,  2.88it/s, Materializing param=model.layers.38.mlp.down_proj.weight]  
Loading weights:  95%|█████████▌| 422/443 [03:23<00:07,  2.88it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 422/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 421/443 [03:23<00:05,  3.75it/s, Materializing param=model.layers.38.input_layernorm.weight]
Loading weights:  95%|█████████▌| 422/443 [03:23<00:05,  3.75it/s, Materializing param=model.layers.38.mlp.down_proj.weight]  
Loading weights:  95%|█████████▌| 422/443 [03:23<00:05,  3.75it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 421/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.38.input_layernorm.weight] 
Loading weights:  95%|█████████▌| 421/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.38.input_layernorm.weight]
Loading weights:  95%|█████████▌| 422/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.38.mlp.down_proj.weight]  
Loading weights:  95%|█████████▌| 422/443 [03:23<00:06,  3.40it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.42it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.49it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.40it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.42it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 422/443 [03:24<00:09,  2.16it/s, Materializing param=model.layers.38.input_layernorm.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:07,  2.58it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.37it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.49it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.42it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.42it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.49it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:09,  2.16it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 422/443 [03:24<00:09,  2.16it/s, Materializing param=model.layers.38.mlp.down_proj.weight]  
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.40it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.37it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 422/443 [03:24<00:09,  2.16it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.40it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.42it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:09,  2.16it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.37it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:08,  2.42it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:09,  2.16it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:09,  2.16it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:09,  2.16it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:07,  2.58it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  95%|█████████▌| 423/443 [03:24<00:07,  2.58it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:11,  1.59it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:13,  1.38it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.55it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight]  
Loading weights:  96%|█████████▌| 424/443 [03:27<00:11,  1.59it/s, Materializing param=model.layers.38.mlp.up_proj.weight]  
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.55it/s, Materializing param=model.layers.38.mlp.up_proj.weight]  
Loading weights:  96%|█████████▌| 424/443 [03:27<00:13,  1.38it/s, Materializing param=model.layers.38.mlp.up_proj.weight]  
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:11,  1.59it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.55it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:13,  1.38it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight]  
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight]  
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight]  
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:12,  1.50it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:11,  1.59it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
Loading weights:  96%|█████████▌| 424/443 [03:27<00:11,  1.59it/s, Materializing param=model.layers.38.mlp.up_proj.weight]  
Loading weights:  96%|█████████▌| 424/443 [03:27<00:11,  1.59it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.59it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.59it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.59it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]        
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.59it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:10,  1.59it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:10,  1.59it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:09,  1.59it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:09,  1.59it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  2.92it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  2.92it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  2.92it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  2.92it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  2.92it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:10,  1.64it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:10,  1.64it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:10,  1.64it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.64it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]        
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.64it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:09,  1.64it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:09,  1.64it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:09,  1.64it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:09,  1.64it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:03,  3.49it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:03,  3.49it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:03,  3.49it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.58it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.62it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.62it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]        
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.58it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.62it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.58it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.62it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]        
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.62it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:11,  1.58it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:12,  1.48it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]        
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:12,  1.48it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]        
Loading weights:  96%|█████████▌| 425/443 [03:27<00:12,  1.48it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:11,  1.48it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]        
Loading weights:  96%|█████████▋| 427/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:11,  1.48it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:10,  1.58it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:10,  1.48it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:10,  1.48it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  3.13it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  3.13it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  3.13it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:10,  1.66it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:10,  1.66it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 425/443 [03:27<00:10,  1.66it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.66it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]        
Loading weights:  96%|█████████▌| 426/443 [03:27<00:10,  1.66it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:09,  1.66it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:09,  1.66it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:10,  1.48it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:10,  1.48it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  98%|█████████▊| 432/443 [03:27<00:03,  3.49it/s, Materializing param=model.layers.39.input_layernorm.weight] 
Loading weights:  98%|█████████▊| 432/443 [03:27<00:03,  3.49it/s, Materializing param=model.layers.39.input_layernorm.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:09,  1.66it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:09,  1.66it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:09,  1.58it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:09,  1.58it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:09,  1.58it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:09,  1.58it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  3.13it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  3.13it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:03,  3.91it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:03,  3.91it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:03,  3.91it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  3.03it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  3.03it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  3.03it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  98%|█████████▊| 432/443 [03:27<00:02,  3.91it/s, Materializing param=model.layers.39.input_layernorm.weight] 
Loading weights:  98%|█████████▊| 432/443 [03:27<00:02,  3.91it/s, Materializing param=model.layers.39.input_layernorm.weight]
Loading weights:  98%|█████████▊| 433/443 [03:27<00:02,  3.91it/s, Materializing param=model.layers.39.mlp.down_proj.weight]  
Loading weights:  98%|█████████▊| 433/443 [03:27<00:02,  3.91it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:07,  2.22it/s, Materializing param=model.layers.38.self_attn.k_norm.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:07,  2.22it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  96%|█████████▋| 427/443 [03:27<00:07,  2.22it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:06,  2.22it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:06,  2.22it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:06,  2.22it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:06,  2.22it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:05,  2.22it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:05,  2.22it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  2.88it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:05,  2.79it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:05,  2.79it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  2.88it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:05,  2.79it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  2.88it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  2.88it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  2.88it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  2.79it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  2.79it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:05,  2.55it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:05,  2.55it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 428/443 [03:27<00:05,  2.55it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:05,  2.55it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:05,  2.55it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:05,  2.55it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:05,  2.55it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  2.90it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  2.90it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 429/443 [03:27<00:04,  2.90it/s, Materializing param=model.layers.38.self_attn.q_norm.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  3.03it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  3.03it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:03,  3.88it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:03,  3.88it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:03,  3.88it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:05,  2.22it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:05,  2.22it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:04,  2.88it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:04,  2.88it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  98%|█████████▊| 432/443 [03:27<00:02,  3.88it/s, Materializing param=model.layers.39.input_layernorm.weight] 
Loading weights:  98%|█████████▊| 432/443 [03:27<00:02,  3.88it/s, Materializing param=model.layers.39.input_layernorm.weight]
Loading weights:  98%|█████████▊| 433/443 [03:27<00:02,  3.88it/s, Materializing param=model.layers.39.mlp.down_proj.weight]  
Loading weights:  98%|█████████▊| 433/443 [03:27<00:02,  3.88it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:04,  2.55it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:04,  2.55it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:04,  2.79it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:04,  2.79it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  2.90it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 430/443 [03:27<00:04,  2.90it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:04,  2.90it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  97%|█████████▋| 431/443 [03:27<00:04,  2.90it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  98%|█████████▊| 432/443 [03:27<00:04,  2.22it/s, Materializing param=model.layers.39.input_layernorm.weight] 
Loading weights:  98%|█████████▊| 432/443 [03:27<00:04,  2.22it/s, Materializing param=model.layers.39.input_layernorm.weight]
Loading weights:  98%|█████████▊| 433/443 [03:27<00:04,  2.22it/s, Materializing param=model.layers.39.mlp.down_proj.weight]  
Loading weights:  98%|█████████▊| 433/443 [03:27<00:04,  2.22it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 432/443 [03:27<00:03,  2.88it/s, Materializing param=model.layers.39.input_layernorm.weight] 
Loading weights:  98%|█████████▊| 432/443 [03:27<00:03,  2.88it/s, Materializing param=model.layers.39.input_layernorm.weight]
Loading weights:  98%|█████████▊| 433/443 [03:27<00:03,  2.88it/s, Materializing param=model.layers.39.mlp.down_proj.weight]  
Loading weights:  98%|█████████▊| 433/443 [03:27<00:03,  2.88it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 432/443 [03:27<00:04,  2.55it/s, Materializing param=model.layers.39.input_layernorm.weight] 
Loading weights:  98%|█████████▊| 432/443 [03:27<00:04,  2.55it/s, Materializing param=model.layers.39.input_layernorm.weight]
Loading weights:  98%|█████████▊| 433/443 [03:27<00:03,  2.55it/s, Materializing param=model.layers.39.mlp.down_proj.weight]  
Loading weights:  98%|█████████▊| 433/443 [03:27<00:03,  2.55it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 432/443 [03:27<00:02,  4.12it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  98%|█████████▊| 432/443 [03:27<00:02,  4.12it/s, Materializing param=model.layers.39.input_layernorm.weight] 
Loading weights:  98%|█████████▊| 432/443 [03:27<00:02,  4.12it/s, Materializing param=model.layers.39.input_layernorm.weight]
Loading weights:  98%|█████████▊| 433/443 [03:27<00:02,  4.12it/s, Materializing param=model.layers.39.mlp.down_proj.weight]  
Loading weights:  98%|█████████▊| 433/443 [03:27<00:02,  4.12it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 432/443 [03:27<00:02,  4.23it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
Loading weights:  98%|█████████▊| 432/443 [03:27<00:02,  4.23it/s, Materializing param=model.layers.39.input_layernorm.weight] 
Loading weights:  98%|█████████▊| 432/443 [03:27<00:02,  4.23it/s, Materializing param=model.layers.39.input_layernorm.weight]
Loading weights:  98%|█████████▊| 433/443 [03:27<00:02,  4.23it/s, Materializing param=model.layers.39.mlp.down_proj.weight]  
Loading weights:  98%|█████████▊| 433/443 [03:27<00:02,  4.23it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.45it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.81it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.45it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.90it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.55it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.40it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.45it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.45it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.81it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.72it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.90it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.55it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.81it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.45it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.40it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.45it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.90it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.55it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.40it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.72it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:03,  2.72it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 433/443 [03:29<00:04,  2.08it/s, Materializing param=model.layers.39.input_layernorm.weight]
Loading weights:  98%|█████████▊| 435/443 [03:29<00:03,  2.55it/s, Materializing param=model.layers.39.mlp.up_proj.weight]  
Loading weights:  98%|█████████▊| 435/443 [03:29<00:03,  2.55it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 433/443 [03:29<00:04,  2.08it/s, Materializing param=model.layers.39.mlp.down_proj.weight]  
Loading weights:  98%|█████████▊| 433/443 [03:29<00:04,  2.08it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:04,  2.08it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 434/443 [03:29<00:04,  2.08it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.57it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.57it/s, Materializing param=model.layers.39.mlp.up_proj.weight]  
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.57it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.33it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.33it/s, Materializing param=model.layers.39.mlp.up_proj.weight]  
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.33it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:02,  2.71it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:02,  2.71it/s, Materializing param=model.layers.39.mlp.up_proj.weight]  
Loading weights:  98%|█████████▊| 435/443 [03:30<00:02,  2.71it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.63it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.63it/s, Materializing param=model.layers.39.mlp.up_proj.weight]  
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.63it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.45it/s, Materializing param=model.layers.39.mlp.up_proj.weight]  
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.45it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.31it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.31it/s, Materializing param=model.layers.39.mlp.up_proj.weight]  
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.31it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.74it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.74it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.74it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.74it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]        
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.74it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:01,  2.74it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:01,  2.74it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.40it/s, Materializing param=model.layers.39.mlp.up_proj.weight]  
Loading weights:  98%|█████████▊| 435/443 [03:30<00:03,  2.40it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  3.97it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  3.97it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  3.97it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.55it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.55it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.55it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.55it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]        
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.55it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:01,  2.55it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:01,  2.55it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.24it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.24it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.24it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.24it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]        
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.24it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.24it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.24it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.55it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.55it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.24it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.24it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.37it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.37it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.19it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.37it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.37it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]        
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.37it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.19it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.37it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.37it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.19it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.19it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]        
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.19it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.19it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.19it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.42it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.42it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:02,  2.42it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.42it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]        
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.42it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.42it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.42it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.19it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.19it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.19it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.19it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.19it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.19it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]        
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.19it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.19it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.19it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.16it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.16it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  98%|█████████▊| 436/443 [03:30<00:03,  2.16it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.16it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]        
Loading weights:  99%|█████████▊| 437/443 [03:30<00:02,  2.16it/s, Materializing param=model.layers.39.self_attn.k_norm.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.16it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 438/443 [03:30<00:02,  2.16it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.37it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.37it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.42it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.42it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.19it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.19it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.16it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 439/443 [03:30<00:01,  2.16it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  3.97it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  4.09it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  3.97it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  4.23it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  4.23it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  4.09it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  3.89it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  4.23it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  4.09it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  3.89it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  3.89it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  4.09it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  4.09it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  3.92it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  4.11it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  4.11it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  4.11it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  3.92it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  4.11it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  4.11it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:00,  3.92it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  3.92it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  3.92it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:01,  2.42it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:01,  2.42it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  4.65it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  4.65it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  4.65it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:01,  2.19it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:01,  2.19it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:01,  2.16it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights:  99%|█████████▉| 440/443 [03:30<00:01,  2.16it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  2.16it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:30<00:00,  2.16it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.83it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:31<00:00,  3.54it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights: 100%|█████████▉| 441/443 [03:31<00:00,  3.54it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.27it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:31<00:00,  3.54it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.91it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  4.65it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.27it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:31<00:00,  3.36it/s, Materializing param=model.layers.39.self_attn.q_norm.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.97it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.66it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.27it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.91it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  4.65it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:31<00:00,  3.36it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.54it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.66it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.91it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.97it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.83it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.54it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 441/443 [03:31<00:00,  3.36it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.66it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.97it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.83it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.36it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|█████████▉| 442/443 [03:31<00:00,  3.36it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  3.83it/s, Materializing param=model.norm.weight]                      
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.04it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.51it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.11it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  3.97it/s, Materializing param=model.norm.weight]                      
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.04it/s, Materializing param=model.norm.weight]                      
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  3.64it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.51it/s, Materializing param=model.norm.weight]                      
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.24it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  3.88it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.04it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.11it/s, Materializing param=model.norm.weight]                      
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  3.97it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.51it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  3.64it/s, Materializing param=model.norm.weight]                      
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.11it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  3.88it/s, Materializing param=model.norm.weight]                      
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  3.83it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  3.64it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.24it/s, Materializing param=model.norm.weight]                      
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  3.88it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  4.24it/s, Materializing param=model.norm.weight]
Loading weights: 100%|██████████| 443/443 [03:31<00:00,  2.09it/s, Materializing param=model.norm.weight]

Loading weights: 100%|██████████| 443/443 [03:31<00:00,  2.09it/s, Materializing param=model.norm.weight]

Loading weights: 100%|██████████| 443/443 [03:31<00:00,  2.09it/s, Materializing param=model.norm.weight]

Loading weights: 100%|██████████| 443/443 [03:31<00:00,  2.09it/s, Materializing param=model.norm.weight]

Loading weights: 100%|██████████| 443/443 [03:31<00:00,  2.09it/s, Materializing param=model.norm.weight]

Loading weights: 100%|██████████| 443/443 [03:31<00:00,  2.09it/s, Materializing param=model.norm.weight]

Loading weights: 100%|██████████| 443/443 [03:31<00:00,  2.09it/s, Materializing param=model.norm.weight]

Loading weights: 100%|██████████| 443/443 [03:31<00:00,  2.09it/s, Materializing param=model.norm.weight]

Downloading builder script: 0.00B [00:00, ?B/s]
Downloading builder script: 9.18kB [00:00, 27.9MB/s]

Downloading extra modules: 0.00B [00:00, ?B/s]
Downloading extra modules: 6.10kB [00:00, 31.6MB/s]

Generating test split:   0%|          | 0/164 [00:00<?, ? examples/s]
Generating test split: 100%|██████████| 164/164 [00:00<00:00, 36447.09 examples/s]

Generating test split:   0%|          | 0/164 [00:00<?, ? examples/s]
Generating test split: 100%|██████████| 164/164 [00:00<00:00, 3259.33 examples/s]

Generating train split:   0%|          | 0/374 [00:00<?, ? examples/s]
Generating train split: 100%|██████████| 374/374 [00:00<00:00, 55912.09 examples/s]

Generating test split:   0%|          | 0/500 [00:00<?, ? examples/s]
Generating test split: 100%|██████████| 500/500 [00:00<00:00, 170264.84 examples/s]

Generating validation split:   0%|          | 0/90 [00:00<?, ? examples/s]
Generating validation split: 100%|██████████| 90/90 [00:00<00:00, 47554.47 examples/s]

Generating prompt split:   0%|          | 0/10 [00:00<?, ? examples/s]
Generating prompt split: 100%|██████████| 10/10 [00:00<00:00, 5596.14 examples/s]

Generating test split:   0%|          | 0/378 [00:00<?, ? examples/s]
Generating test split: 100%|██████████| 378/378 [00:00<00:00, 18723.24 examples/s]
2026-04-30:10:58:47 INFO     [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:47 INFO     [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:47 INFO     [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:47 INFO     [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:47 INFO     [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:47 INFO     [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:47 INFO     [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:47 INFO     [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:47 INFO     [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:47 INFO     [api.task:312] Building contexts for humaneval on rank 5...

  0%|          | 0/20 [00:00<?, ?it/s]
100%|██████████| 20/20 [00:00<00:00, 1323.69it/s]
2026-04-30:10:58:50 INFO     [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:50 INFO     [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:50 INFO     [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:50 INFO     [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:50 INFO     [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:50 INFO     [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:50 INFO     [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:50 INFO     [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:50 INFO     [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:50 INFO     [api.task:312] Building contexts for humaneval on rank 0...

  0%|          | 0/21 [00:00<?, ?it/s]
100%|██████████| 21/21 [00:00<00:00, 1198.91it/s]
ywang29-p4d-debug-2-worker-0:39120:39120 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39120:39120 [0] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39120:39120 [0] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39125:39125 [5] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39120:39120 [0] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39125:39125 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39125:39125 [5] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39125:39125 [5] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39120:39120 [0] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-2-worker-0:39125:39125 [5] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)

[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39120:40089 [0] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol

[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39120:40089 [0] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed

[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39120:40089 [0] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed

[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39125:40091 [5] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol

[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39125:40091 [5] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed

[2026-04-30 10:58:50] ywang29-p4d-debug-2-worker-0:39125:40091 [5] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO ncclCommInitRankConfig comm 0x55cf3325aa10 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO ncclCommInitRankConfig comm 0x5584194aa180 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 commId 0xdd4996e97ba7d4db - Init START
2026-04-30:10:58:51 INFO     [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [api.task:312] Building contexts for humaneval on rank 7...

  0%|          | 0/20 [00:00<?, ?it/s]
100%|██████████| 20/20 [00:00<00:00, 1294.30it/s]
ywang29-p4d-debug-2-worker-0:39127:39127 [7] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39127:39127 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39127:39127 [7] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39127:39127 [7] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39127:39127 [7] NCCL INFO Comm config Blocking set to 1
2026-04-30:10:58:51 INFO     [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [api.task:312] Building contexts for humaneval on rank 3...

  0%|          | 0/21 [00:00<?, ?it/s]
100%|██████████| 21/21 [00:00<00:00, 1287.93it/s]
ywang29-p4d-debug-2-worker-0:39123:39123 [3] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39123:39123 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39123:39123 [3] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39123:39123 [3] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39123:39123 [3] NCCL INFO Comm config Blocking set to 1
2026-04-30:10:58:51 INFO     [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [api.task:312] Building contexts for humaneval on rank 4...

  0%|          | 0/20 [00:00<?, ?it/s]
100%|██████████| 20/20 [00:00<00:00, 1297.48it/s]
ywang29-p4d-debug-2-worker-0:39124:39124 [4] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39124:39124 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39124:39124 [4] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39124:39124 [4] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39124:39124 [4] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39127:40099 [7] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39127:40099 [7] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39127:40099 [7] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO ncclCommInitRankConfig comm 0x5570936d9f40 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39123:40101 [3] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39123:40101 [3] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39123:40101 [3] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO ncclCommInitRankConfig comm 0x55ea64d13050 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39124:40105 [4] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39124:40105 [4] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39124:40105 [4] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO ncclCommInitRankConfig comm 0x557c784ec2c0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO RAS client listening socket at ::1<28028>
2026-04-30:10:58:51 INFO     [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [api.task:312] Building contexts for humaneval on rank 2...

  0%|          | 0/21 [00:00<?, ?it/s]
100%|██████████| 21/21 [00:00<00:00, 1314.48it/s]
ywang29-p4d-debug-2-worker-0:39122:39122 [2] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39122:39122 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39122:39122 [2] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39122:39122 [2] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39122:39122 [2] NCCL INFO Comm config Blocking set to 1
2026-04-30:10:58:51 INFO     [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [api.task:312] Building contexts for humaneval on rank 1...

  0%|          | 0/21 [00:00<?, ?it/s]
100%|██████████| 21/21 [00:00<00:00, 1289.42it/s]
ywang29-p4d-debug-2-worker-0:39121:39121 [1] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39121:39121 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39121:39121 [1] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39121:39121 [1] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39121:39121 [1] NCCL INFO Comm config Blocking set to 1
2026-04-30:10:58:51 INFO     [evaluator_utils:446] Selected tasks:
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval (humaneval/humaneval.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: humaneval_plus (humaneval/humaneval_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp (mbpp/mbpp.yaml)
2026-04-30:10:58:51 INFO     [evaluator_utils:480] Task: mbpp_plus (mbpp/mbpp_plus.yaml)
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] humaneval_plus: Using gen_kwargs: {'until': ['\nclass', '\ndef', '\n#', '\nif', '\nprint'], 'max_gen_toks': 1024, 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [evaluator:307] mbpp_plus: Using gen_kwargs: {'until': ['[DONE]'], 'do_sample': False}
2026-04-30:10:58:51 INFO     [api.task:312] Building contexts for humaneval on rank 6...

  0%|          | 0/20 [00:00<?, ?it/s]ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)

100%|██████████| 20/20 [00:00<00:00, 1191.07it/s]

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39122:40113 [2] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39122:40113 [2] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39122:40113 [2] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39126:39126 [6] NCCL INFO cudaDriverVersion 13000
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39126:39126 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39126:39126 [6] NCCL INFO Bootstrap: Using eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39126:39126 [6] NCCL INFO NCCL version 2.27.7+cuda13.0
ywang29-p4d-debug-2-worker-0:39126:39126 [6] NCCL INFO Comm config Blocking set to 1
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO ncclCommInitRankConfig comm 0x556db8b8dd00 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39121:40117 [1] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39121:40117 [1] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39121:40117 [1] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO ncclCommInitRankConfig comm 0x556f2a64bf80 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Plugin name set by env to libnccl-net.so
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Loaded net plugin Libfabric (v10)
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v10 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v9 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v8 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v7 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Successfully loaded external plugin libnccl-net.so
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Initializing aws-ofi-nccl 1.17.1
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Using Libfabric version 2.3
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Using CUDA driver version 13000 with runtime 13000
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Configuring AWS-specific options
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Setting provider_filter to efa
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Running on p4de.24xlarge platform, topology file /opt/amazon/ofi-nccl/share/aws-ofi-nccl/xml/p4de-24xl-topo.xml
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Internode latency set at 75.0 us
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI Using transport protocol SENDRECV (platform set)

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39126:40121 [6] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):208 NCCL WARN NET/OFI Failed to initialize sendrecv protocol

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39126:40121 [6] int nccl_net_ofi_create_plugin(nccl_net_ofi_plugin_t**):353 NCCL WARN NET/OFI aws-ofi-nccl initialization failed

[2026-04-30 10:58:51] ywang29-p4d-debug-2-worker-0:39126:40121 [6] ncclResult_t nccl_net_ofi_init_v6(ncclDebugLogger_t):162 NCCL WARN NET/OFI Initializing plugin failed
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/IB : No device found.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/IB : Using [RO]; OOB eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NCCL_SOCKET_IFNAME set by environment to eth
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/Socket : Using [0]eth0:10.200.143.174<0>
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Initialized NET plugin Socket
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Assigned NET plugin Socket to comm
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Using network Socket
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO ncclCommInitRankConfig comm 0x559361b695d0 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 commId 0xdd4996e97ba7d4db - Init START
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO RAS client listening socket at ::1<28028>
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Bootstrap timings total 0.000963 (create 0.000041, send 0.000076, recv 0.000220, ring 0.000207, delay 0.000002)
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Bootstrap timings total 0.596544 (create 0.000045, send 0.000095, recv 0.000129, ring 0.000120, delay 0.000001)
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Bootstrap timings total 0.821776 (create 0.000040, send 0.000156, recv 0.820908, ring 0.000117, delay 0.000001)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Bootstrap timings total 0.181152 (create 0.000046, send 0.000079, recv 0.000191, ring 0.050217, delay 0.000001)
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Bootstrap timings total 0.459399 (create 0.000043, send 0.000077, recv 0.000260, ring 0.458590, delay 0.000001)
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Bootstrap timings total 0.821808 (create 0.000040, send 0.000179, recv 0.770821, ring 0.050200, delay 0.000002)
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Bootstrap timings total 0.570657 (create 0.000038, send 0.000069, recv 0.111426, ring 0.180242, delay 0.000001)
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Bootstrap timings total 0.051126 (create 0.000047, send 0.000088, recv 0.000246, ring 0.050217, delay 0.000002)
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Setting affinity for GPU 3 to 0-23,48-71
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NVLS multicast support is not available on dev 3 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Setting affinity for GPU 2 to 0-23,48-71
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NVLS multicast support is not available on dev 2 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Setting affinity for GPU 4 to 24-47,72-95
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NVLS multicast support is not available on dev 4 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Setting affinity for GPU 1 to 0-23,48-71
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NVLS multicast support is not available on dev 1 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Setting affinity for GPU 5 to 24-47,72-95
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NVLS multicast support is not available on dev 5 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Setting affinity for GPU 0 to 0-23,48-71
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NVLS multicast support is not available on dev 0 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Setting affinity for GPU 6 to 24-47,72-95
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NVLS multicast support is not available on dev 6 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Setting affinity for GPU 7 to 24-47,72-95
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NVLS multicast support is not available on dev 7 (NVLS_NCHANNELS 0)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO comm 0x556db8b8dd00 rank 2 nRanks 8 nNodes 1 localRanks 8 localRank 2 MNNVL 0
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO comm 0x559361b695d0 rank 6 nRanks 8 nNodes 1 localRanks 8 localRank 6 MNNVL 0
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO comm 0x556f2a64bf80 rank 1 nRanks 8 nNodes 1 localRanks 8 localRank 1 MNNVL 0
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO comm 0x5570936d9f40 rank 7 nRanks 8 nNodes 1 localRanks 8 localRank 7 MNNVL 0
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO comm 0x5584194aa180 rank 5 nRanks 8 nNodes 1 localRanks 8 localRank 5 MNNVL 0
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO comm 0x55cf3325aa10 rank 0 nRanks 8 nNodes 1 localRanks 8 localRank 0 MNNVL 0
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO comm 0x55ea64d13050 rank 3 nRanks 8 nNodes 1 localRanks 8 localRank 3 MNNVL 0
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO comm 0x557c784ec2c0 rank 4 nRanks 8 nNodes 1 localRanks 8 localRank 4 MNNVL 0
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 00/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Trees [0] 2/-1/-1->1->0 [1] 2/-1/-1->1->0 [2] 2/-1/-1->1->0 [3] 2/-1/-1->1->0 [4] 2/-1/-1->1->0 [5] 2/-1/-1->1->0 [6] 2/-1/-1->1->0 [7] 2/-1/-1->1->0 [8] 2/-1/-1->1->0 [9] 2/-1/-1->1->0 [10] 2/-1/-1->1->0 [11] 2/-1/-1->1->0 [12] 2/-1/-1->1->0 [13] 2/-1/-1->1->0 [14] 2/-1/-1->1->0 [15] 2/-1/-1->1->0 [16] 2/-1/-1->1->0 [17] 2/-1/-1->1->0 [18] 2/-1/-1->1->0 [19] 2/-1/-1->1->0 [20] 2/-1/-1->1->0 [21] 2/-1/-1->1->0 [22] 2/-1/-1->1->0 [23] 2/-1/-1->1->0
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Trees [0] 3/-1/-1->2->1 [1] 3/-1/-1->2->1 [2] 3/-1/-1->2->1 [3] 3/-1/-1->2->1 [4] 3/-1/-1->2->1 [5] 3/-1/-1->2->1 [6] 3/-1/-1->2->1 [7] 3/-1/-1->2->1 [8] 3/-1/-1->2->1 [9] 3/-1/-1->2->1 [10] 3/-1/-1->2->1 [11] 3/-1/-1->2->1 [12] 3/-1/-1->2->1 [13] 3/-1/-1->2->1 [14] 3/-1/-1->2->1 [15] 3/-1/-1->2->1 [16] 3/-1/-1->2->1 [17] 3/-1/-1->2->1 [18] 3/-1/-1->2->1 [19] 3/-1/-1->2->1 [20] 3/-1/-1->2->1 [21] 3/-1/-1->2->1 [22] 3/-1/-1->2->1 [23] 3/-1/-1->2->1
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 01/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 02/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 03/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 04/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4 [2] 6/-1/-1->5->4 [3] 6/-1/-1->5->4 [4] 6/-1/-1->5->4 [5] 6/-1/-1->5->4 [6] 6/-1/-1->5->4 [7] 6/-1/-1->5->4 [8] 6/-1/-1->5->4 [9] 6/-1/-1->5->4 [10] 6/-1/-1->5->4 [11] 6/-1/-1->5->4 [12] 6/-1/-1->5->4 [13] 6/-1/-1->5->4 [14] 6/-1/-1->5->4 [15] 6/-1/-1->5->4 [16] 6/-1/-1->5->4 [17] 6/-1/-1->5->4 [18] 6/-1/-1->5->4 [19] 6/-1/-1->5->4 [20] 6/-1/-1->5->4 [21] 6/-1/-1->5->4 [22] 6/-1/-1->5->4 [23] 6/-1/-1->5->4
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 05/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6 [2] -1/-1/-1->7->6 [3] -1/-1/-1->7->6 [4] -1/-1/-1->7->6 [5] -1/-1/-1->7->6 [6] -1/-1/-1->7->6 [7] -1/-1/-1->7->6 [8] -1/-1/-1->7->6 [9] -1/-1/-1->7->6 [10] -1/-1/-1->7->6 [11] -1/-1/-1->7->6 [12] -1/-1/-1->7->6 [13] -1/-1/-1->7->6 [14] -1/-1/-1->7->6 [15] -1/-1/-1->7->6 [16] -1/-1/-1->7->6 [17] -1/-1/-1->7->6 [18] -1/-1/-1->7->6 [19] -1/-1/-1->7->6 [20] -1/-1/-1->7->6 [21] -1/-1/-1->7->6 [22] -1/-1/-1->7->6 [23] -1/-1/-1->7->6
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5 [2] 7/-1/-1->6->5 [3] 7/-1/-1->6->5 [4] 7/-1/-1->6->5 [5] 7/-1/-1->6->5 [6] 7/-1/-1->6->5 [7] 7/-1/-1->6->5 [8] 7/-1/-1->6->5 [9] 7/-1/-1->6->5 [10] 7/-1/-1->6->5 [11] 7/-1/-1->6->5 [12] 7/-1/-1->6->5 [13] 7/-1/-1->6->5 [14] 7/-1/-1->6->5 [15] 7/-1/-1->6->5 [16] 7/-1/-1->6->5 [17] 7/-1/-1->6->5 [18] 7/-1/-1->6->5 [19] 7/-1/-1->6->5 [20] 7/-1/-1->6->5 [21] 7/-1/-1->6->5 [22] 7/-1/-1->6->5 [23] 7/-1/-1->6->5
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 06/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 07/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Trees [0] 4/-1/-1->3->2 [1] 4/-1/-1->3->2 [2] 4/-1/-1->3->2 [3] 4/-1/-1->3->2 [4] 4/-1/-1->3->2 [5] 4/-1/-1->3->2 [6] 4/-1/-1->3->2 [7] 4/-1/-1->3->2 [8] 4/-1/-1->3->2 [9] 4/-1/-1->3->2 [10] 4/-1/-1->3->2 [11] 4/-1/-1->3->2 [12] 4/-1/-1->3->2 [13] 4/-1/-1->3->2 [14] 4/-1/-1->3->2 [15] 4/-1/-1->3->2 [16] 4/-1/-1->3->2 [17] 4/-1/-1->3->2 [18] 4/-1/-1->3->2 [19] 4/-1/-1->3->2 [20] 4/-1/-1->3->2 [21] 4/-1/-1->3->2 [22] 4/-1/-1->3->2 [23] 4/-1/-1->3->2
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Trees [0] 5/-1/-1->4->3 [1] 5/-1/-1->4->3 [2] 5/-1/-1->4->3 [3] 5/-1/-1->4->3 [4] 5/-1/-1->4->3 [5] 5/-1/-1->4->3 [6] 5/-1/-1->4->3 [7] 5/-1/-1->4->3 [8] 5/-1/-1->4->3 [9] 5/-1/-1->4->3 [10] 5/-1/-1->4->3 [11] 5/-1/-1->4->3 [12] 5/-1/-1->4->3 [13] 5/-1/-1->4->3 [14] 5/-1/-1->4->3 [15] 5/-1/-1->4->3 [16] 5/-1/-1->4->3 [17] 5/-1/-1->4->3 [18] 5/-1/-1->4->3 [19] 5/-1/-1->4->3 [20] 5/-1/-1->4->3 [21] 5/-1/-1->4->3 [22] 5/-1/-1->4->3 [23] 5/-1/-1->4->3
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 08/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 09/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 10/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 11/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 12/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 13/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 14/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 15/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 16/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 17/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 18/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 19/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 20/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 21/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 22/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Channel 23/24 : 0 1 2 3 4 5 6 7
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] 1/-1/-1->0->-1 [2] 1/-1/-1->0->-1 [3] 1/-1/-1->0->-1 [4] 1/-1/-1->0->-1 [5] 1/-1/-1->0->-1 [6] 1/-1/-1->0->-1 [7] 1/-1/-1->0->-1 [8] 1/-1/-1->0->-1 [9] 1/-1/-1->0->-1 [10] 1/-1/-1->0->-1 [11] 1/-1/-1->0->-1 [12] 1/-1/-1->0->-1 [13] 1/-1/-1->0->-1 [14] 1/-1/-1->0->-1 [15] 1/-1/-1->0->-1 [16] 1/-1/-1->0->-1 [17] 1/-1/-1->0->-1 [18] 1/-1/-1->0->-1 [19] 1/-1/-1->0->-1 [20] 1/-1/-1->0->-1 [21] 1/-1/-1->0->-1 [22] 1/-1/-1->0->-1 [23] 1/-1/-1->0->-1
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO P2P Chunksize set to 524288
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so. 
ywang29-p4d-debug-2-worker-0:39126:40130 [6] NCCL INFO [Proxy Service UDS] Device 6 CPU core 26
ywang29-p4d-debug-2-worker-0:39126:40129 [6] NCCL INFO [Proxy Service] Device 6 CPU core 90
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so. 
ywang29-p4d-debug-2-worker-0:39124:40132 [4] NCCL INFO [Proxy Service UDS] Device 4 CPU core 95
ywang29-p4d-debug-2-worker-0:39124:40131 [4] NCCL INFO [Proxy Service] Device 4 CPU core 37
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so. 
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so. 
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so. 
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Check P2P Type isAllDirectP2p 1 directMode 0
ywang29-p4d-debug-2-worker-0:39125:40135 [5] NCCL INFO [Proxy Service UDS] Device 5 CPU core 88
ywang29-p4d-debug-2-worker-0:39125:40133 [5] NCCL INFO [Proxy Service] Device 5 CPU core 91
ywang29-p4d-debug-2-worker-0:39122:40134 [2] NCCL INFO [Proxy Service] Device 2 CPU core 6
ywang29-p4d-debug-2-worker-0:39122:40136 [2] NCCL INFO [Proxy Service UDS] Device 2 CPU core 4
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Proxy Service] Device 0 CPU core 68
ywang29-p4d-debug-2-worker-0:39120:40138 [0] NCCL INFO [Proxy Service UDS] Device 0 CPU core 12
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so. 
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so. 
ywang29-p4d-debug-2-worker-0:39127:40142 [7] NCCL INFO [Proxy Service UDS] Device 7 CPU core 36
ywang29-p4d-debug-2-worker-0:39121:40141 [1] NCCL INFO [Proxy Service UDS] Device 1 CPU core 10
ywang29-p4d-debug-2-worker-0:39121:40140 [1] NCCL INFO [Proxy Service] Device 1 CPU core 18
ywang29-p4d-debug-2-worker-0:39127:40139 [7] NCCL INFO [Proxy Service] Device 7 CPU core 25
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO PROFILER/Plugin: Could not find: libnccl-profiler.so. 
ywang29-p4d-debug-2-worker-0:39123:40143 [3] NCCL INFO [Proxy Service] Device 3 CPU core 5
ywang29-p4d-debug-2-worker-0:39123:40144 [3] NCCL INFO [Proxy Service UDS] Device 3 CPU core 3
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO CC Off, workFifoBytes 1048576
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO threadThresholds 8/8/64 | 64/8/64 | 512 | 512
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO 24 coll channels, 24 collnet channels, 0 nvls channels, 32 p2p channels, 32 p2p channels per peer
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO ncclCommInitRankConfig comm 0x5584194aa180 rank 5 nranks 8 cudaDev 5 nvmlDev 5 busId 901d0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO ncclCommInitRankConfig comm 0x559361b695d0 rank 6 nranks 8 cudaDev 6 nvmlDev 6 busId a01c0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO ncclCommInitRankConfig comm 0x557c784ec2c0 rank 4 nranks 8 cudaDev 4 nvmlDev 4 busId 901c0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO TUNER/Plugin: Could not find: libnccl-tuner.so. Using internal tuner plugin.
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39126:40121 [6] NCCL INFO Init timings - ncclCommInitRankConfig: rank 6 nranks 8 total 0.35 (kernels 0.17, alloc 0.02, bootstrap 0.00, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.02)
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39124:40105 [4] NCCL INFO Init timings - ncclCommInitRankConfig: rank 4 nranks 8 total 0.81 (kernels 0.17, alloc 0.02, bootstrap 0.46, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.07, rest 0.02)
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39125:40091 [5] NCCL INFO Init timings - ncclCommInitRankConfig: rank 5 nranks 8 total 1.20 (kernels 0.18, alloc 0.04, bootstrap 0.82, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.03)
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO ncclCommInitRankConfig comm 0x556f2a64bf80 rank 1 nranks 8 cudaDev 1 nvmlDev 1 busId 101d0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39121:40117 [1] NCCL INFO Init timings - ncclCommInitRankConfig: rank 1 nranks 8 total 0.40 (kernels 0.17, alloc 0.02, bootstrap 0.05, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.02)
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO ncclCommInitRankConfig comm 0x5570936d9f40 rank 7 nranks 8 cudaDev 7 nvmlDev 7 busId a01d0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO ncclCommInitRankConfig comm 0x55ea64d13050 rank 3 nranks 8 cudaDev 3 nvmlDev 3 busId 201d0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39127:40099 [7] NCCL INFO Init timings - ncclCommInitRankConfig: rank 7 nranks 8 total 0.94 (kernels 0.17, alloc 0.02, bootstrap 0.60, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.03)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39123:40101 [3] NCCL INFO Init timings - ncclCommInitRankConfig: rank 3 nranks 8 total 0.92 (kernels 0.17, alloc 0.02, bootstrap 0.57, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.05, rest 0.03)
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO TUNER/Plugin: Failed to find ncclTunerPlugin_v4 symbol.
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO TUNER/Plugin: Using tuner plugin nccl_ofi_tuner
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO ncclCommInitRankConfig comm 0x556db8b8dd00 rank 2 nranks 8 cudaDev 2 nvmlDev 2 busId 201c0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39122:40113 [2] NCCL INFO Init timings - ncclCommInitRankConfig: rank 2 nranks 8 total 0.54 (kernels 0.17, alloc 0.03, bootstrap 0.18, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.03)
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO NET/OFI NCCL_OFI_TUNER is not available for platform : p4de.24xlarge, Fall back to NCCL's tuner
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO ncclCommInitRankConfig comm 0x55cf3325aa10 rank 0 nranks 8 cudaDev 0 nvmlDev 0 busId 101c0 commId 0xdd4996e97ba7d4db - Init COMPLETE
ywang29-p4d-debug-2-worker-0:39120:40089 [0] NCCL INFO Init timings - ncclCommInitRankConfig: rank 0 nranks 8 total 1.20 (kernels 0.18, alloc 0.05, bootstrap 0.82, allgathers 0.00, topo 0.06, graphs 0.00, connections 0.06, rest 0.03)
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 00/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 01/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 00/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 01/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 02/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 03/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 00/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 00/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 02/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 01/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 03/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 01/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 04/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 04/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 02/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 02/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 03/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 05/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 05/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 00/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 03/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 06/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 06/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 04/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 00/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 04/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 07/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 05/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 07/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 05/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 08/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 08/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 01/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 06/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 02/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 00/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 01/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 07/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 09/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 06/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 01/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 10/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 09/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 02/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 08/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 03/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 03/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 11/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 09/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 10/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 07/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 10/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 12/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 04/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 02/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 11/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 00/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 05/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 03/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 04/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 08/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 12/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 13/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 09/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 13/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 01/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 11/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 14/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 06/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 04/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 07/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 05/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 12/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 14/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 15/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 02/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 10/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 13/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 05/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 06/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 15/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 16/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 08/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 07/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 06/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 03/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 17/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 16/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 11/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 12/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 18/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 08/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 07/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 09/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 14/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 17/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 04/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 10/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 08/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 13/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 09/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 15/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 18/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 14/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 05/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 10/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 16/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 19/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 19/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 11/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 15/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 09/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 12/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 06/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 20/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 20/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 17/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 07/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 10/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 16/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 11/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 21/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 18/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 21/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 13/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 11/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 08/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 14/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 22/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 22/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 19/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 17/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 12/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 15/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 20/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Channel 23/0 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Channel 23/0 : 2[2] -> 3[3] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 12/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 21/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 09/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 16/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 17/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 13/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 18/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 22/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 10/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 14/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 13/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 19/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 14/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 18/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 11/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 15/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Channel 23/0 : 4[4] -> 5[5] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 12/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 16/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 19/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 20/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 13/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 17/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 20/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 15/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 21/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 21/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 18/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 14/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 22/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 16/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 15/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 22/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 17/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Channel 23/0 : 6[6] -> 7[7] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 16/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 19/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Channel 23/0 : 0[0] -> 1[1] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 18/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 17/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 20/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 19/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 18/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 21/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 20/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 19/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 22/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 21/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 20/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 22/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Channel 23/0 : 5[5] -> 6[6] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 21/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 22/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Channel 23/0 : 1[1] -> 2[2] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Channel 23/0 : 3[3] -> 4[4] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:40146 [7] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39126:40151 [6] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39125:40152 [5] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39124:40148 [4] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39123:40149 [3] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39122:40150 [2] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39121:40147 [1] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
ywang29-p4d-debug-2-worker-0:39120:40145 [0] NCCL INFO Connected all rings, use ring PXN 0 GDR 1
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for humaneval_plus on rank 0...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for humaneval_plus on rank 5...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for humaneval_plus on rank 3...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for humaneval_plus on rank 6...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for humaneval_plus on rank 2...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for humaneval_plus on rank 1...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for humaneval_plus on rank 4...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for humaneval_plus on rank 7...

  0%|          | 0/21 [00:00<?, ?it/s]
  0%|          | 0/20 [00:00<?, ?it/s]
  0%|          | 0/20 [00:00<?, ?it/s]
  0%|          | 0/20 [00:00<?, ?it/s]
  0%|          | 0/21 [00:00<?, ?it/s]
  0%|          | 0/21 [00:00<?, ?it/s]
  0%|          | 0/20 [00:00<?, ?it/s]
  0%|          | 0/21 [00:00<?, ?it/s]
100%|██████████| 20/20 [00:00<00:00, 1295.60it/s]

100%|██████████| 20/20 [00:00<00:00, 1282.50it/s]

100%|██████████| 20/20 [00:00<00:00, 1278.17it/s]

100%|██████████| 20/20 [00:00<00:00, 1301.55it/s]

100%|██████████| 21/21 [00:00<00:00, 1291.46it/s]

100%|██████████| 21/21 [00:00<00:00, 1267.62it/s]

100%|██████████| 21/21 [00:00<00:00, 1240.32it/s]
100%|██████████| 21/21 [00:00<00:00, 1264.45it/s]

2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for mbpp on rank 4...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for mbpp on rank 5...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for mbpp on rank 3...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for mbpp on rank 7...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for mbpp on rank 1...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for mbpp on rank 2...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for mbpp on rank 6...
2026-04-30:10:58:52 INFO     [api.task:312] Building contexts for mbpp on rank 0...

  0%|          | 0/63 [00:00<?, ?it/s]
  0%|          | 0/63 [00:00<?, ?it/s]
  0%|          | 0/63 [00:00<?, ?it/s]
  0%|          | 0/63 [00:00<?, ?it/s]
  0%|          | 0/62 [00:00<?, ?it/s]
  0%|          | 0/62 [00:00<?, ?it/s]
  0%|          | 0/62 [00:00<?, ?it/s]
  0%|          | 0/62 [00:00<?, ?it/s]
 23%|██▎       | 14/62 [00:00<00:00, 139.42it/s]
 22%|██▏       | 14/63 [00:00<00:00, 138.47it/s]
 22%|██▏       | 14/63 [00:00<00:00, 136.64it/s]
 23%|██▎       | 14/62 [00:00<00:00, 136.75it/s]
 23%|██▎       | 14/62 [00:00<00:00, 136.39it/s]
 22%|██▏       | 14/63 [00:00<00:00, 135.45it/s]
 22%|██▏       | 14/63 [00:00<00:00, 135.64it/s]
 23%|██▎       | 14/62 [00:00<00:00, 134.46it/s]
 44%|████▍     | 28/63 [00:00<00:00, 137.70it/s]
 44%|████▍     | 28/63 [00:00<00:00, 137.84it/s]
 45%|████▌     | 28/62 [00:00<00:00, 137.75it/s]
 45%|████▌     | 28/62 [00:00<00:00, 137.02it/s]
 45%|████▌     | 28/62 [00:00<00:00, 136.82it/s]
 44%|████▍     | 28/63 [00:00<00:00, 136.19it/s]
 47%|████▋     | 29/62 [00:00<00:00, 141.63it/s]
 46%|████▌     | 29/63 [00:00<00:00, 140.40it/s]
 67%|██████▋   | 42/63 [00:00<00:00, 138.58it/s]
 68%|██████▊   | 42/62 [00:00<00:00, 138.44it/s]
 67%|██████▋   | 42/63 [00:00<00:00, 137.98it/s]
 68%|██████▊   | 42/62 [00:00<00:00, 137.68it/s]
 68%|██████▊   | 42/62 [00:00<00:00, 137.67it/s]
 67%|██████▋   | 42/63 [00:00<00:00, 136.69it/s]
 71%|███████   | 44/62 [00:00<00:00, 141.75it/s]
 70%|██████▉   | 44/63 [00:00<00:00, 140.41it/s]
 89%|████████▉ | 56/63 [00:00<00:00, 138.85it/s]
 90%|█████████ | 56/62 [00:00<00:00, 138.66it/s]
 89%|████████▉ | 56/63 [00:00<00:00, 137.96it/s]
 90%|█████████ | 56/62 [00:00<00:00, 137.92it/s]
 90%|█████████ | 56/62 [00:00<00:00, 137.75it/s]
 89%|████████▉ | 56/63 [00:00<00:00, 136.68it/s]
 95%|█████████▌| 59/62 [00:00<00:00, 142.23it/s]
 94%|█████████▎| 59/63 [00:00<00:00, 140.92it/s]
100%|██████████| 62/62 [00:00<00:00, 141.84it/s]

100%|██████████| 63/63 [00:00<00:00, 140.57it/s]

100%|██████████| 62/62 [00:00<00:00, 138.37it/s]

100%|██████████| 62/62 [00:00<00:00, 137.66it/s]

100%|██████████| 62/62 [00:00<00:00, 137.51it/s]

100%|██████████| 63/63 [00:00<00:00, 138.48it/s]

100%|██████████| 63/63 [00:00<00:00, 137.79it/s]

100%|██████████| 63/63 [00:00<00:00, 136.47it/s]
2026-04-30:10:58:53 INFO     [api.task:312] Building contexts for mbpp_plus on rank 4...
2026-04-30:10:58:53 INFO     [api.task:312] Building contexts for mbpp_plus on rank 7...
2026-04-30:10:58:53 INFO     [api.task:312] Building contexts for mbpp_plus on rank 5...
2026-04-30:10:58:53 INFO     [api.task:312] Building contexts for mbpp_plus on rank 3...
2026-04-30:10:58:53 INFO     [api.task:312] Building contexts for mbpp_plus on rank 1...
2026-04-30:10:58:53 INFO     [api.task:312] Building contexts for mbpp_plus on rank 2...
2026-04-30:10:58:53 INFO     [api.task:312] Building contexts for mbpp_plus on rank 6...
2026-04-30:10:58:53 INFO     [api.task:312] Building contexts for mbpp_plus on rank 0...

  0%|          | 0/48 [00:00<?, ?it/s]
  0%|          | 0/47 [00:00<?, ?it/s]
  0%|          | 0/48 [00:00<?, ?it/s]
  0%|          | 0/47 [00:00<?, ?it/s]
  0%|          | 0/47 [00:00<?, ?it/s]
  0%|          | 0/47 [00:00<?, ?it/s]
  0%|          | 0/47 [00:00<?, ?it/s]
  0%|          | 0/47 [00:00<?, ?it/s]
 28%|██▊       | 13/47 [00:00<00:00, 125.27it/s]
 27%|██▋       | 13/48 [00:00<00:00, 124.07it/s]
 28%|██▊       | 13/47 [00:00<00:00, 121.29it/s]
 28%|██▊       | 13/47 [00:00<00:00, 121.41it/s]
 28%|██▊       | 13/47 [00:00<00:00, 120.55it/s]
 28%|██▊       | 13/47 [00:00<00:00, 120.35it/s]
 28%|██▊       | 13/47 [00:00<00:00, 120.84it/s]
 27%|██▋       | 13/48 [00:00<00:00, 120.22it/s]
 55%|█████▌    | 26/47 [00:00<00:00, 124.44it/s]
 54%|█████▍    | 26/48 [00:00<00:00, 123.36it/s]
 55%|█████▌    | 26/47 [00:00<00:00, 121.89it/s]
 55%|█████▌    | 26/47 [00:00<00:00, 121.67it/s]
 55%|█████▌    | 26/47 [00:00<00:00, 121.25it/s]
 55%|█████▌    | 26/47 [00:00<00:00, 120.65it/s]
 54%|█████▍    | 26/48 [00:00<00:00, 120.51it/s]
 55%|█████▌    | 26/47 [00:00<00:00, 119.51it/s]
 83%|████████▎ | 39/47 [00:00<00:00, 123.77it/s]
 81%|████████▏ | 39/48 [00:00<00:00, 122.78it/s]
 83%|████████▎ | 39/47 [00:00<00:00, 121.97it/s]
 83%|████████▎ | 39/47 [00:00<00:00, 121.77it/s]
 83%|████████▎ | 39/47 [00:00<00:00, 121.41it/s]
 83%|████████▎ | 39/47 [00:00<00:00, 120.67it/s]
 81%|████████▏ | 39/48 [00:00<00:00, 120.44it/s]
 83%|████████▎ | 39/47 [00:00<00:00, 119.90it/s]
100%|██████████| 47/47 [00:00<00:00, 123.88it/s]

100%|██████████| 47/47 [00:00<00:00, 121.89it/s]

100%|██████████| 47/47 [00:00<00:00, 121.70it/s]

100%|██████████| 47/47 [00:00<00:00, 121.34it/s]

100%|██████████| 47/47 [00:00<00:00, 120.68it/s]

100%|██████████| 48/48 [00:00<00:00, 122.76it/s]

100%|██████████| 47/47 [00:00<00:00, 119.92it/s]

100%|██████████| 48/48 [00:00<00:00, 120.33it/s]
2026-04-30:10:58:53 INFO     [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO     [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO     [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO     [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO     [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO     [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO     [evaluator:575] Running generate_until requests
2026-04-30:10:58:53 INFO     [evaluator:575] Running generate_until requests
Passed argument batch_size = auto. Detecting largest batch size
Passed argument batch_size = auto. Detecting largest batch sizePassed argument batch_size = auto. Detecting largest batch size
Passed argument batch_size = auto. Detecting largest batch size

Passed argument batch_size = auto. Detecting largest batch sizePassed argument batch_size = auto. Detecting largest batch size

Passed argument batch_size = auto. Detecting largest batch size

Running generate_until requests:   0%|          | 0/153 [00:00<?, ?it/s]Passed argument batch_size = auto. Detecting largest batch size
Determined Largest batch size: 11
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
Determined Largest batch size: 11
Determined Largest batch size: 11
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
Determined Largest batch size: 11
Determined Largest batch size: 11
Determined Largest batch size: 11
Determined Largest batch size: 11
Determined Largest batch size: 11
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
The following generation flags are not valid and may be ignored: ['temperature', 'top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.

Running generate_until requests:   1%|          | 1/153 [06:43<17:02:16, 403.53s/it]
Running generate_until requests:   8%|▊         | 12/153 [07:43<1:08:50, 29.29s/it] 
Running generate_until requests:  15%|█▌        | 23/153 [08:43<33:49, 15.61s/it]  
Running generate_until requests:  22%|██▏       | 34/153 [08:54<17:52,  9.01s/it]
Running generate_until requests:  28%|██▊       | 43/153 [09:05<11:36,  6.33s/it]
Running generate_until requests:  35%|███▌      | 54/153 [09:11<06:51,  4.16s/it]
Running generate_until requests:  42%|████▏     | 65/153 [09:16<04:13,  2.88s/it]
Running generate_until requests:  50%|████▉     | 76/153 [09:23<02:45,  2.14s/it]
Running generate_until requests:  57%|█████▋    | 87/153 [09:31<01:49,  1.67s/it]
Running generate_until requests:  64%|██████▍   | 98/153 [09:35<01:09,  1.27s/it]
Running generate_until requests:  71%|███████   | 109/153 [09:42<00:46,  1.06s/it]2026-04-30:11:08:37 WARNING  [models.huggingface:1038] Left truncation applied. Original sequence length was 4271, truncating to last 3840 tokens. Some content will be lost.

Running generate_until requests:  78%|███████▊  | 120/153 [09:48<00:29,  1.11it/s]
Running generate_until requests:  86%|████████▌ | 131/153 [09:55<00:18,  1.20it/s]
Running generate_until requests:  93%|█████████▎| 142/153 [10:00<00:07,  1.41it/s]
Running generate_until requests: 100%|██████████| 153/153 [10:01<00:00,  1.94it/s]
Running generate_until requests: 100%|██████████| 153/153 [10:01<00:00,  3.93s/it]
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 00/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 00/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 00/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 00/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 00/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 00/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 00/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 01/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 01/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 01/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 01/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 01/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 01/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 01/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 02/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 02/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 02/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 02/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 02/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 02/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 03/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 03/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 03/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 02/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 03/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 03/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 04/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 03/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 04/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 04/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 04/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 03/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 04/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 05/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 04/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 05/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 05/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 04/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 05/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 05/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 05/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 06/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 06/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 05/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 06/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 07/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 07/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 06/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 06/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 07/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 06/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 08/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 07/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 06/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 07/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 08/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 07/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 08/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 09/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 10/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 08/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 08/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 07/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 08/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 09/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 08/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 10/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 11/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 09/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 09/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 09/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 10/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 10/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 09/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 11/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 12/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 10/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 12/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 13/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 09/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 11/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 12/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 10/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 14/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 10/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 11/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 12/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 11/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 11/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 11/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 15/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 16/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 13/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 13/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 12/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 12/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 14/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 13/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 12/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 17/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 13/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 13/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 14/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 14/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 15/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 14/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 18/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 15/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 14/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 15/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 19/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 13/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 15/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 16/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 16/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 14/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 16/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 17/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 15/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 16/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 17/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 15/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 18/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 16/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 17/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 17/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 17/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 19/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 20/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 16/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 18/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 18/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 17/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 21/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 20/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 18/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 19/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 18/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 22/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 18/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 21/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 22/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 20/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 23/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 19/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 19/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 19/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 20/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 23/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 20/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 19/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 20/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 24/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 21/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 20/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 25/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 21/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 21/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 24/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 25/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 21/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 26/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 22/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 22/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 21/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 22/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 26/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 27/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 23/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 23/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 24/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 22/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 22/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 23/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 24/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 28/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 25/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 24/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 25/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 29/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 23/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 27/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 25/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 26/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 24/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 23/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 30/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39121:57210 [1] NCCL INFO Channel 31/1 : 1[1] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 26/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 28/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 25/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 24/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 27/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 26/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 25/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 28/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 26/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 27/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 27/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 29/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 26/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 29/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 28/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 27/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 30/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39125:57211 [5] NCCL INFO Channel 31/1 : 5[5] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 27/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 30/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 29/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 28/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 28/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 30/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39124:57207 [4] NCCL INFO Channel 31/1 : 4[4] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 29/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 28/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 29/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 29/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 30/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39126:57208 [6] NCCL INFO Channel 31/1 : 6[6] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 30/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 30/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39123:57206 [3] NCCL INFO Channel 31/1 : 3[3] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39127:57209 [7] NCCL INFO Channel 31/1 : 7[7] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39122:57212 [2] NCCL INFO Channel 31/1 : 2[2] -> 0[0] via P2P/CUMEM/read
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 4
ywang29-p4d-debug-2-worker-0:39123:40143 [3] NCCL INFO [Service thread] Connection closed by localRank 4
ywang29-p4d-debug-2-worker-0:39125:40133 [0] NCCL INFO [Service thread] Connection closed by localRank 4
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 2
ywang29-p4d-debug-2-worker-0:39123:40143 [3] NCCL INFO [Service thread] Connection closed by localRank 2
ywang29-p4d-debug-2-worker-0:39121:40140 [1] NCCL INFO [Service thread] Connection closed by localRank 2
ywang29-p4d-debug-2-worker-0:39125:40133 [0] NCCL INFO [Service thread] Connection closed by localRank 6
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 6
ywang29-p4d-debug-2-worker-0:39127:40139 [7] NCCL INFO [Service thread] Connection closed by localRank 6
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 5
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 1
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 3
ywang29-p4d-debug-2-worker-0:39120:40137 [0] NCCL INFO [Service thread] Connection closed by localRank 7
2026-04-30:11:12:27 INFO     [_cli.run:449] Logging to W&B failed: 'def pass_at_k(references: list[str], predictions: list[list[str]], k: list[int] = None):\n    global compute_\n    assert k is not None\n    if isinstance(k, int):\n        k = [k]\n    res = compute_.compute(\n        references=references,\n        predictions=predictions,\n        k=k,\n    )\n    return res[0]\n'
2026-04-30:11:12:27 INFO     [loggers.evaluation_tracker:247] Saving results aggregated
2026-04-30:11:12:27 INFO     [loggers.evaluation_tracker:119] Saving per-task samples to saves/qwen3_14b/coding/fft/eval_coding/fft/saves__qwen3_14b__coding__fft/*.jsonl
hf ({'pretrained': 'saves/qwen3_14b/coding/fft', 'dtype': 'bfloat16', 'parallelize': False, 'max_length': 4096}), gen_kwargs: ({}), limit: None, num_fewshot: None, batch_size: auto
|    Tasks     |Version|  Filter   |n-shot| Metric  |   |Value |   |Stderr|
|--------------|------:|-----------|-----:|---------|---|-----:|---|-----:|
|humaneval     |      1|create_test|     0|pass@1   |↑  |0.7439|±  |0.0342|
|humaneval_plus|      1|create_test|     0|pass@1   |↑  |0.6890|±  |0.0363|
|mbpp          |      1|none       |     3|pass_at_1|↑  |0.7060|±  |0.0204|
|mbpp_plus     |      1|none       |     3|pass_at_1|↑  |0.8333|±  |0.0192|

wandb: updating run metadata; uploading artifact run-jm4o6xvg-evaluationeval_results-1S5ClQ; uploading artifact results
wandb: uploading artifact run-jm4o6xvg-evaluationeval_results-1S5ClQ; uploading artifact results
wandb: uploading artifact results
wandb: 
wandb: Run history:
wandb:             humaneval/pass@1,create_test ▁
wandb:      humaneval/pass@1_stderr,create_test ▁
wandb:                     humaneval/sample_len ▁
wandb:        humaneval_plus/pass@1,create_test ▁
wandb: humaneval_plus/pass@1_stderr,create_test ▁
wandb:                humaneval_plus/sample_len ▁
wandb:                           mbpp/pass_at_1 ▁
wandb:                    mbpp/pass_at_1_stderr ▁
wandb:                          mbpp/sample_len ▁
wandb:                      mbpp_plus/pass_at_1 ▁
wandb:                                       +2 ...
wandb: 
wandb: Run summary:
wandb:                          humaneval/alias humaneval
wandb:                           humaneval/name humaneval
wandb:             humaneval/pass@1,create_test 0.7439
wandb:      humaneval/pass@1_stderr,create_test 0.03419
wandb:                     humaneval/sample_len 164
wandb:                     humaneval_plus/alias humaneval_plus
wandb:                      humaneval_plus/name humaneval_plus
wandb:        humaneval_plus/pass@1,create_test 0.68902
wandb: humaneval_plus/pass@1_stderr,create_test 0.03626
wandb:                humaneval_plus/sample_len 164
wandb:                                      +10 ...
wandb: 
wandb: 🚀 View run fft_coding at: https://wandb.ai/maskmoe/MFT-LM-eval/runs/jm4o6xvg
wandb: ⭐️ View project at: https://wandb.ai/maskmoe/MFT-LM-eval
wandb: Synced 5 W&B file(s), 1 media file(s), 4 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20260430_105344-jm4o6xvg/logs