61 lines
79 KiB
Plaintext
61 lines
79 KiB
Plaintext
2026-02-27:11:42:00 INFO [_cli.run:376] Selected Tasks: ['kobest_boolq', 'kobest_copa', 'haerae_general_knowledge', 'haerae_history', 'paws_ko']
|
|
2026-02-27:11:42:00 INFO [evaluator:211] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
|
|
2026-02-27:11:42:00 INFO [evaluator:236] Initializing hf model, with arguments: {'pretrained': '/PROJECT/0325120031_A/ghong/taketimes/llm-bang/outputs/hf_checkpoint-best', 'dtype': 'float16'}
|
|
2026-02-27:11:42:02 INFO [models.huggingface:161] Using device 'cuda:0'
|
|
2026-02-27:11:42:03 INFO [models.huggingface:423] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}
|
|
|
|
Loading weights: 0%| | 0/219 [00:00<?, ?it/s]
|
|
Loading weights: 0%| | 1/219 [00:00<00:00, 9939.11it/s, Materializing param=lm_head.weight]
|
|
Loading weights: 0%| | 1/219 [00:00<00:00, 6168.09it/s, Materializing param=lm_head.weight]
|
|
Loading weights: 1%| | 2/219 [00:03<06:14, 1.73s/it, Materializing param=lm_head.weight]
|
|
Loading weights: 1%| | 2/219 [00:03<06:14, 1.73s/it, Materializing param=model.embed_tokens.weight]
|
|
Loading weights: 1%| | 2/219 [00:03<06:14, 1.73s/it, Materializing param=model.embed_tokens.weight]
|
|
Loading weights: 1%|▏ | 3/219 [00:03<03:50, 1.07s/it, Materializing param=model.embed_tokens.weight]
|
|
Loading weights: 1%|▏ | 3/219 [00:03<03:50, 1.07s/it, Materializing param=model.layers.0.input_layernorm.weight]
|
|
Loading weights: 1%|▏ | 3/219 [00:03<03:50, 1.07s/it, Materializing param=model.layers.0.input_layernorm.weight]
|
|
Loading weights: 2%|▏ | 4/219 [00:03<03:49, 1.07s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
|
|
Loading weights: 2%|▏ | 4/219 [00:03<03:49, 1.07s/it, Materializing param=model.layers.0.mlp.down_proj.weight]
|
|
Loading weights: 2%|▏ | 5/219 [00:03<03:48, 1.07s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
|
Loading weights: 2%|▏ | 5/219 [00:03<03:48, 1.07s/it, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
|
Loading weights: 3%|▎ | 6/219 [00:03<03:47, 1.07s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
|
|
Loading weights: 3%|▎ | 6/219 [00:03<03:47, 1.07s/it, Materializing param=model.layers.0.mlp.up_proj.weight]
|
|
Loading weights: 3%|▎ | 7/219 [00:03<03:46, 1.07s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
|
Loading weights: 3%|▎ | 7/219 [00:03<03:46, 1.07s/it, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
|
Loading weights: 4%|▎ | 8/219 [00:03<03:45, 1.07s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
|
Loading weights: 4%|▎ | 8/219 [00:03<03:45, 1.07s/it, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
|
Loading weights: 4%|▍ | 9/219 [00:03<03:44, 1.07s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
|
Loading weights: 4%|▍ | 9/219 [00:03<03:44, 1.07s/it, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
|
Loading weights: 5%|▍ | 10/219 [00:03<03:43, 1.07s/it, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
|
Loading weights: 5%|▍ | 10/219 [00:03<03:43, 1.07s/it, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
|
Loading weights: 5%|▌ | 11/219 [00:03<03:41, 1.07s/it, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
|
Loading weights: 5%|▌ | 11/219 [00:03<03:41, 1.07s/it, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
|
Loading weights: 5%|▌ | 12/219 [00:03<00:38, 5.35it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
|
Loading weights: 5%|▌ | 12/219 [00:03<00:38, 5.35it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
|
Loading weights: 5%|▌ | 12/219 [00:03<00:38, 5.35it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
|
Loading weights: 6%|▌ | 13/219 [00:03<00:38, 5.35it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
|
Loading weights: 6%|▌ | 13/219 [00:03<00:38, 5.35it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
|
Loading weights: 6%|▋ | 14/219 [00:04<00:38, 5.35it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
|
Loading weights: 6%|▋ | 14/219 [00:04<00:38, 5.35it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
|
Loading weights: 7%|▋ | 15/219 [00:04<00:43, 4.71it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
|
Loading weights: 7%|▋ | 15/219 [00:04<00:43, 4.71it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
|
Loading weights: 7%|▋ | 15/219 [00:04<00:43, 4.71it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
|
Loading weights: 7%|▋ | 16/219 [00:04<00:43, 4.71it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
|
Loading weights: 7%|▋ | 16/219 [00:04<00:43, 4.71it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
|
Loading weights: 8%|▊ | 17/219 [00:04<00:42, 4.71it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
|
Loading weights: 8%|▊ | 17/219 [00:04<00:42, 4.71it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
|
Loading weights: 8%|▊ | 18/219 [00:04<00:42, 4.71it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
|
Loading weights: 8%|▊ | 18/219 [00:04<00:42, 4.71it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
|
Loading weights: 9%|▊ | 19/219 [00:04<00:42, 4.71it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
|
Loading weights: 9%|▊ | 19/219 [00:04<00:42, 4.71it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
|
Loading weights: 9%|▉ | 20/219 [00:04<00:42, 4.71it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
|
Loading weights: 9%|▉ | 20/219 [00:04<00:42, 4.71it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
|
Loading weights: 10%|▉ | 21/219 [00:04<00:27, 7.18it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
|
Loading weights: 10%|▉ | 21/219 [00:04<00:27, 7.18it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
|
Loading weights: 10%|▉ | 21/219 [00:04<00:27, 7.18it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
|
Loading weights: 10%|█ | 22/219 [00:04<00:27, 7.18it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
|
Loading weights: 10%|█ | 22/219 [00:04<00:27, 7.18it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
|
Loading weights: 11%|█ | 23/219 [00:05<00:26, 7.46it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
|
Loading weights: 11%|█ | 23/219 [00:05<00:26, 7.46it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
|
Loading weights: 11%|█ | 23/219 [00:05<00:26, 7.46it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
|
Loading weights: 11%|█ | 24/219 [00:05<00:26, 7.46it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
|
Loading weights: 11%|█ | 24/219 [00:05<00:26, 7.46it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
|
Loading weights: 11%|█▏ | 25/219 [00:05<00:25, 7.46it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
|
Loading weights: 11%|█▏ | 25/219 [00:05<00:25, 7.46it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
|
Loading weights: 12%|█▏ | 26/219 [00:05<00:25, 7.46it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
|
Loading weights: 12%|█▏ | 26/219 [00:05<00:25, 7.46it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
|
Loading weights: 12%|█▏ | 27/219 [00:05<00:25, 7.46it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
|
Loading weights: 12%|█▏ | 27/219 [00:05<00:25, 7.46it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
|
Loading weights: 13%|█▎ | 28/219 [00:05<00:17, 10.79it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
|
Loading weights: 13%|█▎ | 28/219 [00:05<00:17, 10.79it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
|
Loading weights: 13%|█▎ | 28/219 [00:05<00:17, 10.79it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
|
Loading weights: 13%|█▎ | 29/219 [00:05<00:17, 10.79it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
|
Loading weights: 13%|█▎ | 29/219 [00:05<00:17, 10.79it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
|
Loading weights: 14%|█▎ | 30/219 [00:05<00:17, 10.79it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
|
Loading weights: 14%|█▎ | 30/219 [00:05<00:17, 10.79it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
|
Loading weights: 14%|█▍ | 31/219 [00:05<00:14, 12.72it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
|
Loading weights: 14%|█▍ | 31/219 [00:05<00:14, 12.72it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
|
Loading weights: 14%|█▍ | 31/219 [00:05<00:14, 12.72it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
|
Loading weights: 15%|█▍ | 32/219 [00:05<00:14, 12.72it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
|
Loading weights: 15%|█▍ | 32/219 [00:05<00:14, 12.72it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
|
Loading weights: 15%|█▌ | 33/219 [00:05<00:14, 12.72it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
|
Loading weights: 15%|█▌ | 33/219 [00:05<00:14, 12.72it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
|
Loading weights: 16%|█▌ | 34/219 [00:05<00:17, 10.86it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
|
Loading weights: 16%|█▌ | 34/219 [00:05<00:17, 10.86it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
|
Loading weights: 16%|█▌ | 34/219 [00:05<00:17, 10.86it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
|
Loading weights: 16%|█▌ | 35/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
|
Loading weights: 16%|█▌ | 35/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
|
Loading weights: 16%|█▋ | 36/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
|
Loading weights: 16%|█▋ | 36/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
|
Loading weights: 17%|█▋ | 37/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
|
Loading weights: 17%|█▋ | 37/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
|
Loading weights: 17%|█▋ | 38/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
|
Loading weights: 17%|█▋ | 38/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
|
Loading weights: 18%|█▊ | 39/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
|
Loading weights: 18%|█▊ | 39/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
|
Loading weights: 18%|█▊ | 40/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
|
Loading weights: 18%|█▊ | 40/219 [00:05<00:16, 10.86it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
|
Loading weights: 19%|█▊ | 41/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
|
Loading weights: 19%|█▊ | 41/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
|
Loading weights: 19%|█▊ | 41/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
|
Loading weights: 19%|█▉ | 42/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
|
Loading weights: 19%|█▉ | 42/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
|
Loading weights: 20%|█▉ | 43/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
|
Loading weights: 20%|█▉ | 43/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
|
Loading weights: 20%|██ | 44/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
|
Loading weights: 20%|██ | 44/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
|
Loading weights: 21%|██ | 45/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
|
Loading weights: 21%|██ | 45/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
|
Loading weights: 21%|██ | 46/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
|
Loading weights: 21%|██ | 46/219 [00:06<00:11, 15.48it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
|
Loading weights: 21%|██▏ | 47/219 [00:06<00:08, 19.77it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
|
Loading weights: 21%|██▏ | 47/219 [00:06<00:08, 19.77it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
|
Loading weights: 21%|██▏ | 47/219 [00:06<00:08, 19.77it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
|
Loading weights: 22%|██▏ | 48/219 [00:06<00:08, 19.77it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
|
Loading weights: 22%|██▏ | 48/219 [00:06<00:08, 19.77it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
|
Loading weights: 22%|██▏ | 49/219 [00:06<00:08, 19.77it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
|
Loading weights: 22%|██▏ | 49/219 [00:06<00:08, 19.77it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
|
Loading weights: 23%|██▎ | 50/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
|
Loading weights: 23%|██▎ | 50/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
|
Loading weights: 23%|██▎ | 50/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
|
Loading weights: 23%|██▎ | 51/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
|
Loading weights: 23%|██▎ | 51/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
|
Loading weights: 24%|██▎ | 52/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
|
Loading weights: 24%|██▎ | 52/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
|
Loading weights: 24%|██▍ | 53/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
|
Loading weights: 24%|██▍ | 53/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
|
Loading weights: 25%|██▍ | 54/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
|
Loading weights: 25%|██▍ | 54/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
|
Loading weights: 25%|██▌ | 55/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
|
Loading weights: 25%|██▌ | 55/219 [00:06<00:08, 19.15it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
|
Loading weights: 26%|██▌ | 56/219 [00:06<00:06, 24.71it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
|
Loading weights: 26%|██▌ | 56/219 [00:06<00:06, 24.71it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
|
Loading weights: 26%|██▌ | 56/219 [00:06<00:06, 24.71it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
|
Loading weights: 26%|██▌ | 57/219 [00:06<00:06, 24.71it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
|
Loading weights: 26%|██▌ | 57/219 [00:06<00:06, 24.71it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
|
Loading weights: 26%|██▋ | 58/219 [00:06<00:06, 24.71it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
|
Loading weights: 26%|██▋ | 58/219 [00:06<00:06, 24.71it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
|
Loading weights: 27%|██▋ | 59/219 [00:06<00:06, 24.71it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
|
Loading weights: 27%|██▋ | 59/219 [00:06<00:06, 24.71it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
|
Loading weights: 27%|██▋ | 60/219 [00:06<00:06, 25.61it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
|
Loading weights: 27%|██▋ | 60/219 [00:06<00:06, 25.61it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
|
Loading weights: 27%|██▋ | 60/219 [00:06<00:06, 25.61it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
|
Loading weights: 28%|██▊ | 61/219 [00:06<00:06, 25.61it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
|
Loading weights: 28%|██▊ | 61/219 [00:06<00:06, 25.61it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
|
Loading weights: 28%|██▊ | 62/219 [00:06<00:06, 25.61it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
|
Loading weights: 28%|██▊ | 62/219 [00:06<00:06, 25.61it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
|
Loading weights: 29%|██▉ | 63/219 [00:06<00:06, 25.61it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
|
Loading weights: 29%|██▉ | 63/219 [00:06<00:06, 25.61it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
|
Loading weights: 29%|██▉ | 64/219 [00:06<00:05, 27.51it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
|
Loading weights: 29%|██▉ | 64/219 [00:06<00:05, 27.51it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
|
Loading weights: 29%|██▉ | 64/219 [00:06<00:05, 27.51it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
|
Loading weights: 30%|██▉ | 65/219 [00:06<00:05, 27.51it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
|
Loading weights: 30%|██▉ | 65/219 [00:06<00:05, 27.51it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
|
Loading weights: 30%|███ | 66/219 [00:06<00:05, 27.51it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
|
Loading weights: 30%|███ | 66/219 [00:06<00:05, 27.51it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
|
Loading weights: 31%|███ | 67/219 [00:06<00:05, 27.51it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
|
Loading weights: 31%|███ | 67/219 [00:06<00:05, 27.51it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
|
Loading weights: 31%|███ | 68/219 [00:06<00:06, 22.81it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
|
Loading weights: 31%|███ | 68/219 [00:06<00:06, 22.81it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
|
Loading weights: 31%|███ | 68/219 [00:06<00:06, 22.81it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
|
Loading weights: 32%|███▏ | 69/219 [00:06<00:06, 22.81it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
|
Loading weights: 32%|███▏ | 69/219 [00:06<00:06, 22.81it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
|
Loading weights: 32%|███▏ | 70/219 [00:07<00:06, 22.81it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
|
Loading weights: 32%|███▏ | 70/219 [00:07<00:06, 22.81it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
|
Loading weights: 32%|███▏ | 71/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
|
Loading weights: 32%|███▏ | 71/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
|
Loading weights: 32%|███▏ | 71/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
|
Loading weights: 33%|███▎ | 72/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
|
Loading weights: 33%|███▎ | 72/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
|
Loading weights: 33%|███▎ | 73/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
|
Loading weights: 33%|███▎ | 73/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
|
Loading weights: 34%|███▍ | 74/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
|
Loading weights: 34%|███▍ | 74/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
|
Loading weights: 34%|███▍ | 75/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
|
Loading weights: 34%|███▍ | 75/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
|
Loading weights: 35%|███▍ | 76/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
|
Loading weights: 35%|███▍ | 76/219 [00:07<00:06, 23.73it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
|
Loading weights: 35%|███▌ | 77/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
|
Loading weights: 35%|███▌ | 77/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
|
Loading weights: 35%|███▌ | 77/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
|
Loading weights: 36%|███▌ | 78/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
|
Loading weights: 36%|███▌ | 78/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
|
Loading weights: 36%|███▌ | 79/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
|
Loading weights: 36%|███▌ | 79/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
|
Loading weights: 37%|███▋ | 80/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
|
Loading weights: 37%|███▋ | 80/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
|
Loading weights: 37%|███▋ | 81/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
|
Loading weights: 37%|███▋ | 81/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
|
Loading weights: 37%|███▋ | 82/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
|
Loading weights: 37%|███▋ | 82/219 [00:07<00:05, 23.94it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
|
Loading weights: 38%|███▊ | 83/219 [00:07<00:04, 28.89it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
|
Loading weights: 38%|███▊ | 83/219 [00:07<00:04, 28.89it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
|
Loading weights: 38%|███▊ | 83/219 [00:07<00:04, 28.89it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
|
Loading weights: 38%|███▊ | 84/219 [00:07<00:04, 28.89it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
|
Loading weights: 38%|███▊ | 84/219 [00:07<00:04, 28.89it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
|
Loading weights: 39%|███▉ | 85/219 [00:07<00:04, 28.89it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
|
Loading weights: 39%|███▉ | 85/219 [00:07<00:04, 28.89it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
|
Loading weights: 39%|███▉ | 86/219 [00:07<00:04, 28.89it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
|
Loading weights: 39%|███▉ | 86/219 [00:07<00:04, 28.89it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
|
Loading weights: 40%|███▉ | 87/219 [00:07<00:05, 24.29it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
|
Loading weights: 40%|███▉ | 87/219 [00:07<00:05, 24.29it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
|
Loading weights: 40%|███▉ | 87/219 [00:07<00:05, 24.29it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
|
Loading weights: 40%|████ | 88/219 [00:07<00:05, 24.29it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
|
Loading weights: 40%|████ | 88/219 [00:07<00:05, 24.29it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
|
Loading weights: 41%|████ | 89/219 [00:07<00:05, 24.29it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
|
Loading weights: 41%|████ | 89/219 [00:07<00:05, 24.29it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
|
Loading weights: 41%|████ | 90/219 [00:07<00:05, 24.29it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
|
Loading weights: 41%|████ | 90/219 [00:07<00:05, 24.29it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
|
Loading weights: 42%|████▏ | 91/219 [00:07<00:05, 24.29it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
|
Loading weights: 42%|████▏ | 91/219 [00:07<00:05, 24.29it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
|
Loading weights: 42%|████▏ | 92/219 [00:07<00:05, 23.81it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
|
Loading weights: 42%|████▏ | 92/219 [00:07<00:05, 23.81it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
|
Loading weights: 42%|████▏ | 92/219 [00:07<00:05, 23.81it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
|
Loading weights: 42%|████▏ | 93/219 [00:07<00:05, 23.81it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
|
Loading weights: 42%|████▏ | 93/219 [00:07<00:05, 23.81it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
|
Loading weights: 43%|████▎ | 94/219 [00:07<00:05, 23.81it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
|
Loading weights: 43%|████▎ | 94/219 [00:07<00:05, 23.81it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
|
Loading weights: 43%|████▎ | 95/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
|
Loading weights: 43%|████▎ | 95/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
|
Loading weights: 43%|████▎ | 95/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
|
Loading weights: 44%|████▍ | 96/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
|
Loading weights: 44%|████▍ | 96/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
|
Loading weights: 44%|████▍ | 97/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
|
Loading weights: 44%|████▍ | 97/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
|
Loading weights: 45%|████▍ | 98/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
|
Loading weights: 45%|████▍ | 98/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
|
Loading weights: 45%|████▌ | 99/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
|
Loading weights: 45%|████▌ | 99/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
|
Loading weights: 46%|████▌ | 100/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
|
Loading weights: 46%|████▌ | 100/219 [00:08<00:06, 18.00it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
|
Loading weights: 46%|████▌ | 101/219 [00:08<00:06, 19.35it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
|
Loading weights: 46%|████▌ | 101/219 [00:08<00:06, 19.35it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
|
Loading weights: 46%|████▌ | 101/219 [00:08<00:06, 19.35it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
|
Loading weights: 47%|████▋ | 102/219 [00:08<00:06, 19.35it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
|
Loading weights: 47%|████▋ | 102/219 [00:08<00:06, 19.35it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
|
Loading weights: 47%|████▋ | 103/219 [00:08<00:05, 19.35it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
|
Loading weights: 47%|████▋ | 103/219 [00:08<00:05, 19.35it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
|
Loading weights: 47%|████▋ | 104/219 [00:09<00:08, 12.95it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
|
Loading weights: 47%|████▋ | 104/219 [00:09<00:08, 12.95it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
|
Loading weights: 47%|████▋ | 104/219 [00:09<00:08, 12.95it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
|
Loading weights: 48%|████▊ | 105/219 [00:09<00:08, 12.95it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
|
Loading weights: 48%|████▊ | 105/219 [00:09<00:08, 12.95it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
|
Loading weights: 48%|████▊ | 106/219 [00:09<00:08, 13.10it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
|
Loading weights: 48%|████▊ | 106/219 [00:09<00:08, 13.10it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
|
Loading weights: 48%|████▊ | 106/219 [00:09<00:08, 13.10it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
|
Loading weights: 49%|████▉ | 107/219 [00:09<00:08, 13.10it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
|
Loading weights: 49%|████▉ | 107/219 [00:09<00:08, 13.10it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
|
Loading weights: 49%|████▉ | 108/219 [00:09<00:08, 13.10it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
|
Loading weights: 49%|████▉ | 108/219 [00:09<00:08, 13.10it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
|
Loading weights: 50%|████▉ | 109/219 [00:09<00:08, 13.10it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
|
Loading weights: 50%|████▉ | 109/219 [00:09<00:08, 13.10it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
|
Loading weights: 50%|█████ | 110/219 [00:09<00:06, 16.67it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
|
Loading weights: 50%|█████ | 110/219 [00:09<00:06, 16.67it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
|
Loading weights: 50%|█████ | 110/219 [00:09<00:06, 16.67it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
|
Loading weights: 51%|█████ | 111/219 [00:09<00:06, 16.67it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
|
Loading weights: 51%|█████ | 111/219 [00:09<00:06, 16.67it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
|
Loading weights: 51%|█████ | 112/219 [00:09<00:06, 16.67it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
|
Loading weights: 51%|█████ | 112/219 [00:09<00:06, 16.67it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
|
Loading weights: 52%|█████▏ | 113/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
|
Loading weights: 52%|█████▏ | 113/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
|
Loading weights: 52%|█████▏ | 113/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
|
Loading weights: 52%|█████▏ | 114/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
|
Loading weights: 52%|█████▏ | 114/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
|
Loading weights: 53%|█████▎ | 115/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
|
Loading weights: 53%|█████▎ | 115/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
|
Loading weights: 53%|█████▎ | 116/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
|
Loading weights: 53%|█████▎ | 116/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
|
Loading weights: 53%|█████▎ | 117/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
|
Loading weights: 53%|█████▎ | 117/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
|
Loading weights: 54%|█████▍ | 118/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
|
Loading weights: 54%|█████▍ | 118/219 [00:09<00:09, 10.63it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
|
Loading weights: 54%|█████▍ | 119/219 [00:10<00:08, 12.40it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
|
Loading weights: 54%|█████▍ | 119/219 [00:10<00:08, 12.40it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
|
Loading weights: 54%|█████▍ | 119/219 [00:10<00:08, 12.40it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
|
Loading weights: 55%|█████▍ | 120/219 [00:10<00:07, 12.40it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
|
Loading weights: 55%|█████▍ | 120/219 [00:10<00:07, 12.40it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
|
Loading weights: 55%|█████▌ | 121/219 [00:10<00:07, 12.40it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
|
Loading weights: 55%|█████▌ | 121/219 [00:10<00:07, 12.40it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
|
Loading weights: 56%|█████▌ | 122/219 [00:10<00:07, 13.78it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
|
Loading weights: 56%|█████▌ | 122/219 [00:10<00:07, 13.78it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
|
Loading weights: 56%|█████▌ | 122/219 [00:10<00:07, 13.78it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
|
Loading weights: 56%|█████▌ | 123/219 [00:10<00:06, 13.78it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
|
Loading weights: 56%|█████▌ | 123/219 [00:10<00:06, 13.78it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
|
Loading weights: 57%|█████▋ | 124/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
|
Loading weights: 57%|█████▋ | 124/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
|
Loading weights: 57%|█████▋ | 124/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
|
Loading weights: 57%|█████▋ | 125/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
|
Loading weights: 57%|█████▋ | 125/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
|
Loading weights: 58%|█████▊ | 126/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
|
Loading weights: 58%|█████▊ | 126/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
|
Loading weights: 58%|█████▊ | 127/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
|
Loading weights: 58%|█████▊ | 127/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
|
Loading weights: 58%|█████▊ | 128/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
|
Loading weights: 58%|█████▊ | 128/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
|
Loading weights: 59%|█████▉ | 129/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
|
Loading weights: 59%|█████▉ | 129/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
|
Loading weights: 59%|█████▉ | 130/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
|
Loading weights: 59%|█████▉ | 130/219 [00:10<00:08, 10.76it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
|
Loading weights: 60%|█████▉ | 131/219 [00:10<00:04, 17.74it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
|
Loading weights: 60%|█████▉ | 131/219 [00:10<00:04, 17.74it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
|
Loading weights: 60%|█████▉ | 131/219 [00:10<00:04, 17.74it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
|
Loading weights: 60%|██████ | 132/219 [00:11<00:04, 17.74it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
|
Loading weights: 60%|██████ | 132/219 [00:11<00:04, 17.74it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
|
Loading weights: 61%|██████ | 133/219 [00:11<00:04, 17.74it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
|
Loading weights: 61%|██████ | 133/219 [00:11<00:04, 17.74it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
|
Loading weights: 61%|██████ | 134/219 [00:11<00:05, 15.45it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
|
Loading weights: 61%|██████ | 134/219 [00:11<00:05, 15.45it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
|
Loading weights: 61%|██████ | 134/219 [00:11<00:05, 15.45it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
|
Loading weights: 62%|██████▏ | 135/219 [00:11<00:05, 15.45it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
|
Loading weights: 62%|██████▏ | 135/219 [00:11<00:05, 15.45it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
|
Loading weights: 62%|██████▏ | 136/219 [00:11<00:05, 15.45it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
|
Loading weights: 62%|██████▏ | 136/219 [00:11<00:05, 15.45it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
|
Loading weights: 63%|██████▎ | 137/219 [00:11<00:05, 15.45it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
|
Loading weights: 63%|██████▎ | 137/219 [00:11<00:05, 15.45it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
|
Loading weights: 63%|██████▎ | 138/219 [00:11<00:05, 15.45it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
|
Loading weights: 63%|██████▎ | 138/219 [00:11<00:05, 15.45it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
|
Loading weights: 63%|██████▎ | 139/219 [00:11<00:04, 18.78it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
|
Loading weights: 63%|██████▎ | 139/219 [00:11<00:04, 18.78it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
|
Loading weights: 63%|██████▎ | 139/219 [00:11<00:04, 18.78it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
|
Loading weights: 64%|██████▍ | 140/219 [00:11<00:04, 18.78it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
|
Loading weights: 64%|██████▍ | 140/219 [00:11<00:04, 18.78it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
|
Loading weights: 64%|██████▍ | 141/219 [00:11<00:04, 18.78it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
|
Loading weights: 64%|██████▍ | 141/219 [00:11<00:04, 18.78it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
|
Loading weights: 65%|██████▍ | 142/219 [00:11<00:06, 12.25it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
|
Loading weights: 65%|██████▍ | 142/219 [00:11<00:06, 12.25it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
|
Loading weights: 65%|██████▍ | 142/219 [00:11<00:06, 12.25it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
|
Loading weights: 65%|██████▌ | 143/219 [00:11<00:06, 12.25it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
|
Loading weights: 65%|██████▌ | 143/219 [00:11<00:06, 12.25it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
|
Loading weights: 66%|██████▌ | 144/219 [00:11<00:06, 12.25it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
|
Loading weights: 66%|██████▌ | 144/219 [00:11<00:06, 12.25it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
|
Loading weights: 66%|██████▌ | 145/219 [00:11<00:06, 12.25it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
|
Loading weights: 66%|██████▌ | 145/219 [00:11<00:06, 12.25it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
|
Loading weights: 67%|██████▋ | 146/219 [00:11<00:05, 12.25it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
|
Loading weights: 67%|██████▋ | 146/219 [00:11<00:05, 12.25it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
|
Loading weights: 67%|██████▋ | 147/219 [00:11<00:05, 12.25it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
|
Loading weights: 67%|██████▋ | 147/219 [00:11<00:05, 12.25it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
|
Loading weights: 68%|██████▊ | 148/219 [00:11<00:05, 12.25it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
|
Loading weights: 68%|██████▊ | 148/219 [00:11<00:05, 12.25it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
|
Loading weights: 68%|██████▊ | 149/219 [00:11<00:03, 18.07it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
|
Loading weights: 68%|██████▊ | 149/219 [00:11<00:03, 18.07it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
|
Loading weights: 68%|██████▊ | 149/219 [00:11<00:03, 18.07it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
|
Loading weights: 68%|██████▊ | 150/219 [00:12<00:03, 18.07it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
|
Loading weights: 68%|██████▊ | 150/219 [00:12<00:03, 18.07it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
|
Loading weights: 69%|██████▉ | 151/219 [00:12<00:03, 18.07it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
|
Loading weights: 69%|██████▉ | 151/219 [00:12<00:03, 18.07it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
|
Loading weights: 69%|██████▉ | 152/219 [00:12<00:04, 13.61it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
|
Loading weights: 69%|██████▉ | 152/219 [00:12<00:04, 13.61it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
|
Loading weights: 69%|██████▉ | 152/219 [00:12<00:04, 13.61it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
|
Loading weights: 70%|██████▉ | 153/219 [00:12<00:04, 13.61it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
|
Loading weights: 70%|██████▉ | 153/219 [00:12<00:04, 13.61it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
|
Loading weights: 70%|███████ | 154/219 [00:12<00:04, 13.61it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
|
Loading weights: 70%|███████ | 154/219 [00:12<00:04, 13.61it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
|
Loading weights: 71%|███████ | 155/219 [00:12<00:04, 13.17it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
|
Loading weights: 71%|███████ | 155/219 [00:12<00:04, 13.17it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
|
Loading weights: 71%|███████ | 155/219 [00:12<00:04, 13.17it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
|
Loading weights: 71%|███████ | 156/219 [00:12<00:04, 13.17it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
|
Loading weights: 71%|███████ | 156/219 [00:12<00:04, 13.17it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
|
Loading weights: 72%|███████▏ | 157/219 [00:12<00:04, 13.17it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
|
Loading weights: 72%|███████▏ | 157/219 [00:12<00:04, 13.17it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
|
Loading weights: 72%|███████▏ | 158/219 [00:12<00:04, 13.17it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
|
Loading weights: 72%|███████▏ | 158/219 [00:12<00:04, 13.17it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
|
Loading weights: 73%|███████▎ | 159/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
|
Loading weights: 73%|███████▎ | 159/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
|
Loading weights: 73%|███████▎ | 159/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
|
Loading weights: 73%|███████▎ | 160/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
|
Loading weights: 73%|███████▎ | 160/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
|
Loading weights: 74%|███████▎ | 161/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
|
Loading weights: 74%|███████▎ | 161/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
|
Loading weights: 74%|███████▍ | 162/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
|
Loading weights: 74%|███████▍ | 162/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
|
Loading weights: 74%|███████▍ | 163/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
|
Loading weights: 74%|███████▍ | 163/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
|
Loading weights: 75%|███████▍ | 164/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
|
Loading weights: 75%|███████▍ | 164/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
|
Loading weights: 75%|███████▌ | 165/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
|
Loading weights: 75%|███████▌ | 165/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
|
Loading weights: 76%|███████▌ | 166/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
|
Loading weights: 76%|███████▌ | 166/219 [00:13<00:05, 10.40it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
|
Loading weights: 76%|███████▋ | 167/219 [00:13<00:04, 10.40it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
|
Loading weights: 76%|███████▋ | 167/219 [00:13<00:04, 10.40it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
|
Loading weights: 77%|███████▋ | 168/219 [00:13<00:03, 13.65it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
|
Loading weights: 77%|███████▋ | 168/219 [00:13<00:03, 13.65it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
|
Loading weights: 77%|███████▋ | 168/219 [00:13<00:03, 13.65it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
|
Loading weights: 77%|███████▋ | 169/219 [00:13<00:03, 13.65it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
|
Loading weights: 77%|███████▋ | 169/219 [00:13<00:03, 13.65it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
|
Loading weights: 78%|███████▊ | 170/219 [00:13<00:03, 12.53it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
|
Loading weights: 78%|███████▊ | 170/219 [00:13<00:03, 12.53it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
|
Loading weights: 78%|███████▊ | 170/219 [00:13<00:03, 12.53it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
|
Loading weights: 78%|███████▊ | 171/219 [00:13<00:03, 12.53it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
|
Loading weights: 78%|███████▊ | 171/219 [00:13<00:03, 12.53it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
|
Loading weights: 79%|███████▊ | 172/219 [00:13<00:03, 12.53it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
|
Loading weights: 79%|███████▊ | 172/219 [00:13<00:03, 12.53it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
|
Loading weights: 79%|███████▉ | 173/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
|
Loading weights: 79%|███████▉ | 173/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
|
Loading weights: 79%|███████▉ | 173/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
|
Loading weights: 79%|███████▉ | 174/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
|
Loading weights: 79%|███████▉ | 174/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
|
Loading weights: 80%|███████▉ | 175/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
|
Loading weights: 80%|███████▉ | 175/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
|
Loading weights: 80%|████████ | 176/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
|
Loading weights: 80%|████████ | 176/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
|
Loading weights: 80%|████████ | 176/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
|
Loading weights: 81%|████████ | 177/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
|
Loading weights: 81%|████████ | 177/219 [00:14<00:03, 11.70it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
|
Loading weights: 81%|████████▏ | 178/219 [00:14<00:04, 9.20it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
|
Loading weights: 81%|████████▏ | 178/219 [00:14<00:04, 9.20it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
|
Loading weights: 81%|████████▏ | 178/219 [00:14<00:04, 9.20it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
|
Loading weights: 82%|████████▏ | 179/219 [00:14<00:04, 9.20it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
|
Loading weights: 82%|████████▏ | 179/219 [00:14<00:04, 9.20it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
|
Loading weights: 82%|████████▏ | 180/219 [00:14<00:04, 9.20it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
|
Loading weights: 82%|████████▏ | 180/219 [00:14<00:04, 9.20it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
|
Loading weights: 83%|████████▎ | 181/219 [00:14<00:04, 9.20it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
|
Loading weights: 83%|████████▎ | 181/219 [00:14<00:04, 9.20it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
|
Loading weights: 83%|████████▎ | 182/219 [00:14<00:04, 9.20it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
|
Loading weights: 83%|████████▎ | 182/219 [00:14<00:04, 9.20it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
|
Loading weights: 84%|████████▎ | 183/219 [00:14<00:03, 9.20it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
|
Loading weights: 84%|████████▎ | 183/219 [00:14<00:03, 9.20it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
|
Loading weights: 84%|████████▍ | 184/219 [00:14<00:03, 9.20it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
|
Loading weights: 84%|████████▍ | 184/219 [00:14<00:03, 9.20it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
|
Loading weights: 84%|████████▍ | 185/219 [00:14<00:03, 9.20it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
|
Loading weights: 84%|████████▍ | 185/219 [00:14<00:03, 9.20it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
|
Loading weights: 85%|████████▍ | 186/219 [00:15<00:02, 12.29it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
|
Loading weights: 85%|████████▍ | 186/219 [00:15<00:02, 12.29it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
|
Loading weights: 85%|████████▍ | 186/219 [00:15<00:02, 12.29it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
|
Loading weights: 85%|████████▌ | 187/219 [00:15<00:02, 12.29it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
|
Loading weights: 85%|████████▌ | 187/219 [00:15<00:02, 12.29it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
|
Loading weights: 86%|████████▌ | 188/219 [00:15<00:02, 12.29it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
|
Loading weights: 86%|████████▌ | 188/219 [00:15<00:02, 12.29it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
|
Loading weights: 86%|████████▋ | 189/219 [00:15<00:02, 12.29it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
|
Loading weights: 86%|████████▋ | 189/219 [00:15<00:02, 12.29it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
|
Loading weights: 87%|████████▋ | 190/219 [00:15<00:02, 12.29it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
|
Loading weights: 87%|████████▋ | 190/219 [00:15<00:02, 12.29it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
|
Loading weights: 87%|████████▋ | 191/219 [00:15<00:02, 11.35it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
|
Loading weights: 87%|████████▋ | 191/219 [00:15<00:02, 11.35it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
|
Loading weights: 87%|████████▋ | 191/219 [00:15<00:02, 11.35it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
|
Loading weights: 88%|████████▊ | 192/219 [00:15<00:02, 11.35it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
|
Loading weights: 88%|████████▊ | 192/219 [00:15<00:02, 11.35it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
|
Loading weights: 88%|████████▊ | 193/219 [00:15<00:02, 11.35it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
|
Loading weights: 88%|████████▊ | 193/219 [00:15<00:02, 11.35it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
|
Loading weights: 89%|████████▊ | 194/219 [00:15<00:02, 11.35it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
|
Loading weights: 89%|████████▊ | 194/219 [00:15<00:02, 11.35it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
|
Loading weights: 89%|████████▉ | 195/219 [00:16<00:01, 13.09it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
|
Loading weights: 89%|████████▉ | 195/219 [00:16<00:01, 13.09it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
|
Loading weights: 89%|████████▉ | 195/219 [00:16<00:01, 13.09it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
|
Loading weights: 89%|████████▉ | 196/219 [00:16<00:01, 13.09it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
|
Loading weights: 89%|████████▉ | 196/219 [00:16<00:01, 13.09it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
|
Loading weights: 90%|████████▉ | 197/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
|
Loading weights: 90%|████████▉ | 197/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
|
Loading weights: 90%|████████▉ | 197/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
|
Loading weights: 90%|█████████ | 198/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
|
Loading weights: 90%|█████████ | 198/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
|
Loading weights: 91%|█████████ | 199/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
|
Loading weights: 91%|█████████ | 199/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
|
Loading weights: 91%|█████████▏| 200/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
|
Loading weights: 91%|█████████▏| 200/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
|
Loading weights: 92%|█████████▏| 201/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
|
Loading weights: 92%|█████████▏| 201/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
|
Loading weights: 92%|█████████▏| 202/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
|
Loading weights: 92%|█████████▏| 202/219 [00:16<00:01, 12.27it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
|
Loading weights: 93%|█████████▎| 203/219 [00:16<00:01, 15.84it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
|
Loading weights: 93%|█████████▎| 203/219 [00:16<00:01, 15.84it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
|
Loading weights: 93%|█████████▎| 203/219 [00:16<00:01, 15.84it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
|
Loading weights: 93%|█████████▎| 204/219 [00:16<00:00, 15.84it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
|
Loading weights: 93%|█████████▎| 204/219 [00:16<00:00, 15.84it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
|
Loading weights: 94%|█████████▎| 205/219 [00:16<00:00, 15.68it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
|
Loading weights: 94%|█████████▎| 205/219 [00:16<00:00, 15.68it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
|
Loading weights: 94%|█████████▎| 205/219 [00:16<00:00, 15.68it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
|
Loading weights: 94%|█████████▍| 206/219 [00:16<00:00, 15.68it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
|
Loading weights: 94%|█████████▍| 206/219 [00:16<00:00, 15.68it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
|
Loading weights: 95%|█████████▍| 207/219 [00:16<00:00, 15.68it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
|
Loading weights: 95%|█████████▍| 207/219 [00:16<00:00, 15.68it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
|
Loading weights: 95%|█████████▍| 208/219 [00:16<00:00, 15.68it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
|
Loading weights: 95%|█████████▍| 208/219 [00:16<00:00, 15.68it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
|
Loading weights: 95%|█████████▌| 209/219 [00:16<00:00, 15.68it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
|
Loading weights: 95%|█████████▌| 209/219 [00:16<00:00, 15.68it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
|
Loading weights: 96%|█████████▌| 210/219 [00:16<00:00, 15.90it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
|
Loading weights: 96%|█████████▌| 210/219 [00:16<00:00, 15.90it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
|
Loading weights: 96%|█████████▌| 210/219 [00:16<00:00, 15.90it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
|
Loading weights: 96%|█████████▋| 211/219 [00:16<00:00, 15.90it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
|
Loading weights: 96%|█████████▋| 211/219 [00:16<00:00, 15.90it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
|
Loading weights: 97%|█████████▋| 212/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
|
Loading weights: 97%|█████████▋| 212/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
|
Loading weights: 97%|█████████▋| 212/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
|
Loading weights: 97%|█████████▋| 213/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
|
Loading weights: 97%|█████████▋| 213/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
|
Loading weights: 98%|█████████▊| 214/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
|
Loading weights: 98%|█████████▊| 214/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
|
Loading weights: 98%|█████████▊| 215/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
|
Loading weights: 98%|█████████▊| 215/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
|
Loading weights: 99%|█████████▊| 216/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
|
Loading weights: 99%|█████████▊| 216/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
|
Loading weights: 99%|█████████▉| 217/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
|
Loading weights: 99%|█████████▉| 217/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
|
Loading weights: 100%|█████████▉| 218/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
|
Loading weights: 100%|█████████▉| 218/219 [00:17<00:00, 14.19it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
|
Loading weights: 100%|██████████| 219/219 [00:17<00:00, 14.19it/s, Materializing param=model.norm.weight]
|
|
Loading weights: 100%|██████████| 219/219 [00:17<00:00, 14.19it/s, Materializing param=model.norm.weight]
|
|
Loading weights: 100%|██████████| 219/219 [00:17<00:00, 12.74it/s, Materializing param=model.norm.weight]
|
|
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
|
|
|
Generating train split: 0%| | 0/3665 [00:00<?, ? examples/s]
|
|
Generating train split: 100%|██████████| 3665/3665 [00:00<00:00, 831689.89 examples/s]
|
|
|
|
Generating test split: 0%| | 0/1404 [00:00<?, ? examples/s]
|
|
Generating test split: 100%|██████████| 1404/1404 [00:00<00:00, 592315.71 examples/s]
|
|
|
|
Generating validation split: 0%| | 0/700 [00:00<?, ? examples/s]
|
|
Generating validation split: 100%|██████████| 700/700 [00:00<00:00, 393831.36 examples/s]
|
|
|
|
Generating train split: 0%| | 0/3076 [00:00<?, ? examples/s]
|
|
Generating train split: 100%|██████████| 3076/3076 [00:00<00:00, 1183531.70 examples/s]
|
|
|
|
Generating test split: 0%| | 0/1000 [00:00<?, ? examples/s]
|
|
Generating test split: 100%|██████████| 1000/1000 [00:00<00:00, 576774.48 examples/s]
|
|
|
|
Generating validation split: 0%| | 0/500 [00:00<?, ? examples/s]
|
|
Generating validation split: 100%|██████████| 500/500 [00:00<00:00, 404075.53 examples/s]
|
|
|
|
Generating test split: 0 examples [00:00, ? examples/s]
|
|
Generating test split: 176 examples [00:00, 77241.55 examples/s]
|
|
|
|
Generating test split: 0 examples [00:00, ? examples/s]
|
|
Generating test split: 188 examples [00:00, 93383.37 examples/s]
|
|
|
|
Generating train split: 0%| | 0/49401 [00:00<?, ? examples/s]
|
|
Generating train split: 100%|██████████| 49401/49401 [00:00<00:00, 2001244.11 examples/s]
|
|
|
|
Generating test split: 0%| | 0/2000 [00:00<?, ? examples/s]
|
|
Generating test split: 100%|██████████| 2000/2000 [00:00<00:00, 899582.63 examples/s]
|
|
|
|
Generating validation split: 0%| | 0/2000 [00:00<?, ? examples/s]
|
|
Generating validation split: 100%|██████████| 2000/2000 [00:00<00:00, 1037423.70 examples/s]
|
|
2026-02-27:11:43:03 INFO [tasks:700] Selected tasks:
|
|
2026-02-27:11:43:03 INFO [tasks:691] Task: paws_ko (paws-x/paws_ko.yaml)
|
|
2026-02-27:11:43:03 INFO [tasks:691] Task: haerae_history (haerae/haerae_hi.yaml)
|
|
2026-02-27:11:43:03 INFO [tasks:691] Task: haerae_general_knowledge (haerae/haerae_gk.yaml)
|
|
2026-02-27:11:43:03 INFO [tasks:691] Task: kobest_copa (kobest/kobest_copa.yaml)
|
|
2026-02-27:11:43:03 INFO [tasks:691] Task: kobest_boolq (kobest/kobest_boolq.yaml)
|
|
2026-02-27:11:43:03 WARNING [evaluator:333] Overwriting default num_fewshot of paws_ko from None to 0
|
|
2026-02-27:11:43:03 WARNING [evaluator:333] Overwriting default num_fewshot of haerae_history from None to 0
|
|
2026-02-27:11:43:03 WARNING [evaluator:333] Overwriting default num_fewshot of haerae_general_knowledge from None to 0
|
|
2026-02-27:11:43:03 WARNING [evaluator:333] Overwriting default num_fewshot of kobest_copa from None to 0
|
|
2026-02-27:11:43:03 WARNING [evaluator:333] Overwriting default num_fewshot of kobest_boolq from None to 0
|
|
2026-02-27:11:43:03 INFO [api.task:311] Building contexts for paws_ko on rank 0...
|
|
|
|
0%| | 0/2000 [00:00<?, ?it/s]
|
|
6%|▋ | 125/2000 [00:00<00:01, 1245.64it/s]
|
|
12%|█▎ | 250/2000 [00:00<00:01, 1244.84it/s]
|
|
19%|█▉ | 375/2000 [00:00<00:01, 1245.80it/s]
|
|
25%|██▌ | 501/2000 [00:00<00:01, 1249.00it/s]
|
|
31%|███▏ | 628/2000 [00:00<00:01, 1253.42it/s]
|
|
38%|███▊ | 755/2000 [00:00<00:00, 1258.99it/s]
|
|
44%|████▍ | 883/2000 [00:00<00:00, 1262.94it/s]
|
|
50%|█████ | 1010/2000 [00:00<00:00, 1264.62it/s]
|
|
57%|█████▋ | 1137/2000 [00:00<00:00, 1265.37it/s]
|
|
63%|██████▎ | 1264/2000 [00:01<00:00, 1250.69it/s]
|
|
70%|██████▉ | 1391/2000 [00:01<00:00, 1254.63it/s]
|
|
76%|███████▌ | 1519/2000 [00:01<00:00, 1260.47it/s]
|
|
82%|████████▏ | 1647/2000 [00:01<00:00, 1265.42it/s]
|
|
89%|████████▉ | 1775/2000 [00:01<00:00, 1267.53it/s]
|
|
95%|█████████▌| 1902/2000 [00:01<00:00, 1266.73it/s]
|
|
100%|██████████| 2000/2000 [00:01<00:00, 1101.53it/s]
|
|
2026-02-27:11:43:04 INFO [api.task:311] Building contexts for haerae_history on rank 0...
|
|
|
|
0%| | 0/188 [00:00<?, ?it/s]
|
|
100%|██████████| 188/188 [00:00<00:00, 3762.73it/s]
|
|
2026-02-27:11:43:05 INFO [api.task:311] Building contexts for haerae_general_knowledge on rank 0...
|
|
|
|
0%| | 0/176 [00:00<?, ?it/s]
|
|
100%|██████████| 176/176 [00:00<00:00, 3825.31it/s]
|
|
2026-02-27:11:43:05 INFO [api.task:311] Building contexts for kobest_copa on rank 0...
|
|
|
|
0%| | 0/1000 [00:00<?, ?it/s]
|
|
100%|██████████| 1000/1000 [00:00<00:00, 195229.19it/s]
|
|
2026-02-27:11:43:05 INFO [api.task:311] Building contexts for kobest_boolq on rank 0...
|
|
|
|
0%| | 0/1404 [00:00<?, ?it/s]
|
|
21%|██ | 298/1404 [00:00<00:00, 2978.72it/s]
|
|
43%|████▎ | 599/1404 [00:00<00:00, 2992.01it/s]
|
|
64%|██████▍ | 899/1404 [00:00<00:00, 2995.38it/s]
|
|
85%|████████▌ | 1199/1404 [00:00<00:00, 2992.41it/s]
|
|
100%|██████████| 1404/1404 [00:00<00:00, 2990.89it/s]
|
|
2026-02-27:11:43:05 INFO [evaluator:584] Running loglikelihood requests
|
|
|
|
Running loglikelihood requests: 0%| | 0/10628 [00:00<?, ?it/s]
|
|
Running loglikelihood requests: 0%| | 1/10628 [00:01<5:23:23, 1.83s/it]
|
|
Running loglikelihood requests: 1%| | 129/10628 [00:01<01:58, 88.80it/s]
|
|
Running loglikelihood requests: 2%|▏ | 257/10628 [00:02<00:55, 187.27it/s]
|
|
Running loglikelihood requests: 4%|▎ | 385/10628 [00:02<00:35, 292.11it/s]
|
|
Running loglikelihood requests: 4%|▍ | 458/10628 [00:02<00:35, 289.81it/s]
|
|
Running loglikelihood requests: 5%|▌ | 577/10628 [00:02<00:25, 395.42it/s]
|
|
Running loglikelihood requests: 7%|▋ | 705/10628 [00:02<00:19, 515.12it/s]
|
|
Running loglikelihood requests: 8%|▊ | 833/10628 [00:02<00:15, 629.77it/s]
|
|
Running loglikelihood requests: 9%|▉ | 961/10628 [00:03<00:13, 736.53it/s]
|
|
Running loglikelihood requests: 10%|█ | 1089/10628 [00:03<00:11, 829.21it/s]
|
|
Running loglikelihood requests: 11%|█▏ | 1217/10628 [00:03<00:10, 904.16it/s]
|
|
Running loglikelihood requests: 13%|█▎ | 1345/10628 [00:03<00:09, 966.04it/s]
|
|
Running loglikelihood requests: 14%|█▍ | 1473/10628 [00:03<00:09, 1015.77it/s]
|
|
Running loglikelihood requests: 15%|█▌ | 1601/10628 [00:03<00:08, 1026.68it/s]
|
|
Running loglikelihood requests: 17%|█▋ | 1793/10628 [00:03<00:07, 1182.59it/s]
|
|
Running loglikelihood requests: 18%|█▊ | 1921/10628 [00:03<00:07, 1180.18it/s]
|
|
Running loglikelihood requests: 20%|█▉ | 2113/10628 [00:03<00:06, 1297.26it/s]
|
|
Running loglikelihood requests: 22%|██▏ | 2305/10628 [00:04<00:06, 1314.13it/s]
|
|
Running loglikelihood requests: 23%|██▎ | 2497/10628 [00:04<00:06, 1331.10it/s]
|
|
Running loglikelihood requests: 25%|██▌ | 2689/10628 [00:04<00:05, 1412.99it/s]
|
|
Running loglikelihood requests: 27%|██▋ | 2881/10628 [00:04<00:05, 1398.27it/s]
|
|
Running loglikelihood requests: 29%|██▉ | 3073/10628 [00:04<00:05, 1466.01it/s]
|
|
Running loglikelihood requests: 31%|███ | 3265/10628 [00:04<00:05, 1449.78it/s]
|
|
Running loglikelihood requests: 33%|███▎ | 3457/10628 [00:04<00:04, 1509.37it/s]
|
|
Running loglikelihood requests: 34%|███▍ | 3649/10628 [00:05<00:04, 1474.97it/s]
|
|
Running loglikelihood requests: 36%|███▌ | 3841/10628 [00:05<00:04, 1464.36it/s]
|
|
Running loglikelihood requests: 38%|███▊ | 4033/10628 [00:05<00:04, 1455.68it/s]
|
|
Running loglikelihood requests: 40%|███▉ | 4225/10628 [00:05<00:04, 1460.91it/s]
|
|
Running loglikelihood requests: 42%|████▏ | 4417/10628 [00:05<00:04, 1464.49it/s]
|
|
Running loglikelihood requests: 43%|████▎ | 4565/10628 [00:05<00:04, 1468.23it/s]
|
|
Running loglikelihood requests: 45%|████▍ | 4737/10628 [00:05<00:04, 1439.85it/s]
|
|
Running loglikelihood requests: 46%|████▋ | 4929/10628 [00:05<00:03, 1467.82it/s]
|
|
Running loglikelihood requests: 48%|████▊ | 5121/10628 [00:06<00:03, 1495.31it/s]
|
|
Running loglikelihood requests: 50%|████▉ | 5313/10628 [00:06<00:03, 1602.51it/s]
|
|
Running loglikelihood requests: 52%|█████▏ | 5505/10628 [00:06<00:03, 1601.03it/s]
|
|
Running loglikelihood requests: 54%|█████▎ | 5697/10628 [00:06<00:03, 1610.21it/s]
|
|
Running loglikelihood requests: 56%|█████▌ | 5928/10628 [00:06<00:02, 1797.07it/s]
|
|
Running loglikelihood requests: 58%|█████▊ | 6145/10628 [00:06<00:02, 1718.71it/s]
|
|
Running loglikelihood requests: 60%|██████ | 6392/10628 [00:06<00:02, 1916.17it/s]
|
|
Running loglikelihood requests: 62%|██████▏ | 6589/10628 [00:06<00:02, 1840.17it/s]
|
|
Running loglikelihood requests: 64%|██████▍ | 6785/10628 [00:06<00:02, 1725.73it/s]
|
|
Running loglikelihood requests: 66%|██████▌ | 6977/10628 [00:07<00:02, 1733.70it/s]
|
|
Running loglikelihood requests: 67%|██████▋ | 7169/10628 [00:07<00:01, 1735.14it/s]
|
|
Running loglikelihood requests: 70%|██████▉ | 7425/10628 [00:07<00:01, 1800.34it/s]
|
|
Running loglikelihood requests: 72%|███████▏ | 7617/10628 [00:07<00:01, 1787.78it/s]
|
|
Running loglikelihood requests: 73%|███████▎ | 7809/10628 [00:07<00:01, 1786.78it/s]
|
|
Running loglikelihood requests: 75%|███████▌ | 8001/10628 [00:07<00:01, 1693.08it/s]
|
|
Running loglikelihood requests: 77%|███████▋ | 8195/10628 [00:07<00:01, 1749.00it/s]
|
|
Running loglikelihood requests: 79%|███████▉ | 8387/10628 [00:07<00:01, 1686.48it/s]
|
|
Running loglikelihood requests: 81%|████████ | 8579/10628 [00:07<00:01, 1641.90it/s]
|
|
Running loglikelihood requests: 83%|████████▎ | 8771/10628 [00:08<00:01, 1652.38it/s]
|
|
Running loglikelihood requests: 84%|████████▍ | 8963/10628 [00:08<00:00, 1717.91it/s]
|
|
Running loglikelihood requests: 88%|████████▊ | 9315/10628 [00:08<00:00, 2214.48it/s]
|
|
Running loglikelihood requests: 91%|█████████ | 9668/10628 [00:08<00:00, 2516.52it/s]
|
|
Running loglikelihood requests: 95%|█████████▍| 10047/10628 [00:08<00:00, 2875.17it/s]
|
|
Running loglikelihood requests: 97%|█████████▋| 10340/10628 [00:08<00:00, 2798.00it/s]
|
|
Running loglikelihood requests: 100%|██████████| 10628/10628 [00:08<00:00, 2386.47it/s]
|
|
Running loglikelihood requests: 100%|██████████| 10628/10628 [00:08<00:00, 1209.96it/s]
|
|
fatal: (현재 폴더 또는 상위 폴더 부터 마운트 위치 /PROJECT 까지 일부가) 깃 저장소가 아닙니다
|
|
파일 시스템 경계에서 중지합니다. (GIT_DISCOVERY_ACROSS_FILESYSTEM 설정되지 않음)
|
|
2026-02-27:11:43:18 INFO [loggers.evaluation_tracker:247] Saving results aggregated
|
|
2026-02-27:11:43:18 INFO [loggers.evaluation_tracker:119] Saving per-task samples to /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/quick_sft_v2_best/__PROJECT__0325120031_A__ghong__taketimes__llm-bang__outputs__hf_checkpoint-best/*.jsonl
|
|
Passed argument batch_size = auto:1. Detecting largest batch size
|
|
Determined largest batch size: 64
|
|
hf ({'pretrained': '/PROJECT/0325120031_A/ghong/taketimes/llm-bang/outputs/hf_checkpoint-best', 'dtype': 'float16'}), gen_kwargs: ({}), limit: None, num_fewshot: 0, batch_size: auto (64)
|
|
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
|
|
|------------------------|------:|------|-----:|--------|---|-----:|---|------|
|
|
|haerae_general_knowledge| 1|none | 0|acc |↑ |0.2273|± |0.0317|
|
|
| | |none | 0|acc_norm|↑ |0.2273|± |0.0317|
|
|
|haerae_history | 1|none | 0|acc |↑ |0.1543|± |0.0264|
|
|
| | |none | 0|acc_norm|↑ |0.1543|± |0.0264|
|
|
|kobest_boolq | 1|none | 0|acc |↑ |0.5000|± |0.0133|
|
|
| | |none | 0|f1 |↑ |0.3346|± | N/A|
|
|
|kobest_copa | 1|none | 0|acc |↑ |0.6460|± |0.0151|
|
|
| | |none | 0|f1 |↑ |0.6455|± | N/A|
|
|
|paws_ko | 1|none | 0|acc |↑ |0.4900|± |0.0112|
|
|
|