15 lines
170 KiB
Plaintext
15 lines
170 KiB
Plaintext
[2026-02-06 15:45:47,242] [WARNING] [huggingface_hub.utils._http._warn_on_warning_headers:779] [PID:7207] Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
[2026-02-06 15:45:48,737] [WARNING] [py.warnings._showwarnmsg:110] [PID:7207] /root/axolotl/.venv/lib/python3.11/site-packages/torch/distributed/distributed_c10d.py:4807: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user.
|
||
warnings.warn( # warn only once
|
||
|
||
|
||
Downloading (incomplete total...): 0.00B [00:00, ?B/s]
|
||
|
||
Fetching 5 files: 0%| | 0/5 [00:00<?, ?it/s][A
|
||
|
||
Fetching 5 files: 20%|████████████████████████▌ | 1/5 [00:30<02:01, 30.46s/it][A
|
||
Fetching 5 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:30<00:00, 6.09s/it]
|
||
|
||
Download complete: : 0.00B [00:30, ?B/s]
|
||
Download complete: : 0.00B [00:30, ?B/s]
|
||
|
||
Loading weights: 0%| | 0/363 [00:00<?, ?it/s]
|
||
Loading weights: 0%|▏ | 1/363 [00:00<00:00, 9731.56it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 0%|▏ | 1/363 [00:00<00:00, 4219.62it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%|▍ | 2/363 [00:00<00:00, 4454.92it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%|▍ | 2/363 [00:00<00:00, 3539.50it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%|▌ | 3/363 [00:00<00:00, 4051.16it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%|▌ | 3/363 [00:00<00:00, 3553.49it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 1%|▋ | 4/363 [00:00<00:00, 3981.30it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 1%|▋ | 4/363 [00:00<00:00, 3622.02it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 1%|▊ | 5/363 [00:00<00:00, 3811.62it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 1%|▊ | 5/363 [00:00<00:00, 3549.68it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 2%|█ | 6/363 [00:00<00:00, 3430.92it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 2%|█ | 6/363 [00:00<00:00, 3243.44it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 2%|█ | 7/363 [00:00<00:00, 3500.67it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|█ | 7/363 [00:00<00:00, 3339.03it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 2%|█▎ | 8/363 [00:00<00:00, 2977.06it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 2%|█▎ | 8/363 [00:00<00:00, 2752.39it/s, Materializing param=model.layers.0.self_attn.k_proj.weight]
|
||
Loading weights: 2%|█▍ | 9/363 [00:00<00:00, 2928.30it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 2%|█▍ | 9/363 [00:00<00:00, 2749.96it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 3%|█▋ | 10/363 [00:00<00:00, 2913.72it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 3%|█▋ | 10/363 [00:00<00:00, 2780.45it/s, Materializing param=model.layers.0.self_attn.q_proj.weight]
|
||
Loading weights: 3%|█▊ | 11/363 [00:00<00:00, 2935.32it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 3%|█▊ | 11/363 [00:00<00:00, 2861.94it/s, Materializing param=model.layers.0.self_attn.v_proj.weight]
|
||
Loading weights: 3%|█▉ | 12/363 [00:00<00:00, 3007.75it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 3%|█▉ | 12/363 [00:00<00:00, 2937.87it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 4%|██▏ | 13/363 [00:00<00:00, 3076.56it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 4%|██▏ | 13/363 [00:00<00:00, 3009.66it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 4%|██▍ | 14/363 [00:00<00:00, 3023.39it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 4%|██▍ | 14/363 [00:00<00:00, 2959.24it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 4%|██▋ | 15/363 [00:00<00:00, 2960.55it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 4%|██▋ | 15/363 [00:00<00:00, 2870.58it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 4%|██▏ | 16/363 [00:00<00:00, 2975.74it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 4%|██▏ | 16/363 [00:00<00:00, 2897.12it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 5%|██▊ | 17/363 [00:00<00:00, 2998.33it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 5%|██▊ | 17/363 [00:00<00:00, 2949.21it/s, Materializing param=model.layers.1.self_attn.k_proj.weight]
|
||
Loading weights: 5%|██▉ | 18/363 [00:00<00:00, 3045.97it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 5%|██▉ | 18/363 [00:00<00:00, 2845.20it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 5%|███ | 19/363 [00:00<00:00, 2905.81it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 5%|███ | 19/363 [00:00<00:00, 2794.34it/s, Materializing param=model.layers.1.self_attn.q_proj.weight]
|
||
Loading weights: 6%|███▎ | 20/363 [00:00<00:00, 2876.16it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 6%|███▎ | 20/363 [00:00<00:00, 2836.86it/s, Materializing param=model.layers.1.self_attn.v_proj.weight]
|
||
Loading weights: 6%|███▍ | 21/363 [00:00<00:00, 2917.63it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 6%|███▍ | 21/363 [00:00<00:00, 2878.82it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 6%|███▊ | 22/363 [00:00<00:00, 2896.62it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 6%|███▊ | 22/363 [00:00<00:00, 2859.55it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 6%|███▉ | 23/363 [00:00<00:00, 2921.62it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 6%|███▉ | 23/363 [00:00<00:00, 2865.30it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 7%|████▏ | 24/363 [00:00<00:00, 2937.44it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 7%|████▏ | 24/363 [00:00<00:00, 2886.90it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 7%|███▌ | 25/363 [00:00<00:00, 2954.57it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 7%|███▌ | 25/363 [00:00<00:00, 2919.52it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 7%|████▏ | 26/363 [00:00<00:00, 2964.01it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 7%|████▏ | 26/363 [00:00<00:00, 2931.35it/s, Materializing param=model.layers.2.self_attn.k_proj.weight]
|
||
Loading weights: 7%|████▍ | 27/363 [00:00<00:00, 2994.98it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 7%|████▍ | 27/363 [00:00<00:00, 2964.02it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 8%|████▌ | 28/363 [00:00<00:00, 2860.15it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 8%|████▌ | 28/363 [00:00<00:00, 2830.92it/s, Materializing param=model.layers.2.self_attn.q_proj.weight]
|
||
Loading weights: 8%|████▋ | 29/363 [00:00<00:00, 2890.70it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 8%|████▋ | 29/363 [00:00<00:00, 2808.40it/s, Materializing param=model.layers.2.self_attn.v_proj.weight]
|
||
Loading weights: 8%|████▉ | 30/363 [00:00<00:00, 2840.00it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 8%|████▉ | 30/363 [00:00<00:00, 2755.42it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 9%|█████▎ | 31/363 [00:00<00:00, 2809.01it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 9%|█████▎ | 31/363 [00:00<00:00, 2784.88it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 9%|█████▍ | 32/363 [00:00<00:00, 2839.32it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 9%|█████▍ | 32/363 [00:00<00:00, 2815.91it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 9%|█████▊ | 33/363 [00:00<00:00, 2865.79it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 9%|█████▊ | 33/363 [00:00<00:00, 2842.78it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 9%|████▊ | 34/363 [00:00<00:00, 2894.09it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 9%|████▊ | 34/363 [00:00<00:00, 2871.19it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 10%|█████▋ | 35/363 [00:00<00:00, 2921.12it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 10%|█████▋ | 35/363 [00:00<00:00, 2897.93it/s, Materializing param=model.layers.3.self_attn.k_proj.weight]
|
||
Loading weights: 10%|█████▊ | 36/363 [00:00<00:00, 2946.70it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 10%|█████▊ | 36/363 [00:00<00:00, 2924.67it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 10%|██████ | 37/363 [00:00<00:00, 2970.36it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 10%|██████ | 37/363 [00:00<00:00, 2947.17it/s, Materializing param=model.layers.3.self_attn.q_proj.weight]
|
||
Loading weights: 10%|██████▏ | 38/363 [00:00<00:00, 2989.92it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 10%|██████▏ | 38/363 [00:00<00:00, 2967.87it/s, Materializing param=model.layers.3.self_attn.v_proj.weight]
|
||
Loading weights: 11%|██████▍ | 39/363 [00:00<00:00, 3012.59it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 11%|██████▍ | 39/363 [00:00<00:00, 2990.89it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 11%|██████▊ | 40/363 [00:00<00:00, 3035.39it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 11%|██████▊ | 40/363 [00:00<00:00, 3014.23it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 11%|███████ | 41/363 [00:00<00:00, 2935.43it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 11%|███████ | 41/363 [00:00<00:00, 2913.90it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 12%|███████▍ | 42/363 [00:00<00:00, 2955.02it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 12%|███████▍ | 42/363 [00:00<00:00, 2928.84it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 12%|██████ | 43/363 [00:00<00:00, 2954.70it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 12%|██████ | 43/363 [00:00<00:00, 2934.85it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 12%|███████▏ | 44/363 [00:00<00:00, 2973.73it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 12%|███████▏ | 44/363 [00:00<00:00, 2949.30it/s, Materializing param=model.layers.4.self_attn.k_proj.weight]
|
||
Loading weights: 12%|███████▎ | 45/363 [00:00<00:00, 2977.64it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 12%|███████▎ | 45/363 [00:00<00:00, 2959.34it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 13%|███████▍ | 46/363 [00:00<00:00, 2997.89it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 13%|███████▍ | 46/363 [00:00<00:00, 2969.46it/s, Materializing param=model.layers.4.self_attn.q_proj.weight]
|
||
Loading weights: 13%|███████▋ | 47/363 [00:00<00:00, 2979.04it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 13%|███████▋ | 47/363 [00:00<00:00, 2960.97it/s, Materializing param=model.layers.4.self_attn.v_proj.weight]
|
||
Loading weights: 13%|███████▉ | 48/363 [00:00<00:00, 2979.17it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 13%|███████▉ | 48/363 [00:00<00:00, 2961.56it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 13%|████████▎ | 49/363 [00:00<00:00, 2996.67it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 13%|████████▎ | 49/363 [00:00<00:00, 2977.92it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 14%|████████▌ | 50/363 [00:00<00:00, 2922.25it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 14%|████████▌ | 50/363 [00:00<00:00, 2905.49it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 14%|████████▉ | 51/363 [00:00<00:00, 2902.91it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 14%|████████▉ | 51/363 [00:00<00:00, 2877.87it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 14%|███████▎ | 52/363 [00:00<00:00, 2909.14it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 14%|███████▎ | 52/363 [00:00<00:00, 2893.47it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 15%|████████▌ | 53/363 [00:00<00:00, 2925.98it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 15%|████████▌ | 53/363 [00:00<00:00, 2910.96it/s, Materializing param=model.layers.5.self_attn.k_proj.weight]
|
||
Loading weights: 15%|████████▊ | 54/363 [00:00<00:00, 2942.99it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 15%|████████▊ | 54/363 [00:00<00:00, 2926.52it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 15%|████████▉ | 55/363 [00:00<00:00, 2957.45it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 15%|████████▉ | 55/363 [00:00<00:00, 2942.43it/s, Materializing param=model.layers.5.self_attn.q_proj.weight]
|
||
Loading weights: 15%|█████████ | 56/363 [00:00<00:00, 2973.63it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 15%|█████████ | 56/363 [00:00<00:00, 2958.72it/s, Materializing param=model.layers.5.self_attn.v_proj.weight]
|
||
Loading weights: 16%|█████████▍ | 57/363 [00:00<00:00, 2988.52it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 16%|█████████▍ | 57/363 [00:00<00:00, 2973.72it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 16%|█████████▉ | 58/363 [00:00<00:00, 3003.03it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 16%|█████████▉ | 58/363 [00:00<00:00, 2988.35it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 16%|██████████ | 59/363 [00:00<00:00, 2920.45it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 16%|██████████ | 59/363 [00:00<00:00, 2906.01it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 17%|██████████▌ | 60/363 [00:00<00:00, 2933.15it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 17%|██████████▌ | 60/363 [00:00<00:00, 2919.10it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 17%|████████▌ | 61/363 [00:00<00:00, 2946.32it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 17%|████████▌ | 61/363 [00:00<00:00, 2930.76it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 17%|██████████ | 62/363 [00:00<00:00, 2775.91it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 17%|██████████ | 62/363 [00:00<00:00, 2721.01it/s, Materializing param=model.layers.6.self_attn.k_proj.weight]
|
||
Loading weights: 17%|██████████▏ | 63/363 [00:00<00:00, 2744.28it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 17%|██████████▏ | 63/363 [00:00<00:00, 2731.40it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 18%|██████████▍ | 64/363 [00:00<00:00, 2756.04it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 18%|██████████▍ | 64/363 [00:00<00:00, 2699.85it/s, Materializing param=model.layers.6.self_attn.q_proj.weight]
|
||
Loading weights: 18%|██████████▌ | 65/363 [00:00<00:00, 2723.03it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 18%|██████████▌ | 65/363 [00:00<00:00, 2710.85it/s, Materializing param=model.layers.6.self_attn.v_proj.weight]
|
||
Loading weights: 18%|██████████▉ | 66/363 [00:00<00:00, 2734.50it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 18%|██████████▉ | 66/363 [00:00<00:00, 2722.61it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 18%|███████████▍ | 67/363 [00:00<00:00, 2745.98it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 18%|███████████▍ | 67/363 [00:00<00:00, 2733.35it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 19%|███████████▌ | 68/363 [00:00<00:00, 2756.18it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 19%|███████████▌ | 68/363 [00:00<00:00, 2744.38it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 19%|████████████▏ | 69/363 [00:00<00:00, 2767.09it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 19%|████████████▏ | 69/363 [00:00<00:00, 2755.52it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 19%|█████████▊ | 70/363 [00:00<00:00, 2777.90it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█████████▊ | 70/363 [00:00<00:00, 2765.96it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 20%|███████████▌ | 71/363 [00:00<00:00, 2787.33it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 20%|███████████▌ | 71/363 [00:00<00:00, 2775.38it/s, Materializing param=model.layers.7.self_attn.k_proj.weight]
|
||
Loading weights: 20%|███████████▋ | 72/363 [00:00<00:00, 2695.26it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 20%|███████████▋ | 72/363 [00:00<00:00, 2679.16it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 20%|███████████▊ | 73/363 [00:00<00:00, 2698.78it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 20%|███████████▊ | 73/363 [00:00<00:00, 2688.14it/s, Materializing param=model.layers.7.self_attn.q_proj.weight]
|
||
Loading weights: 20%|████████████ | 74/363 [00:00<00:00, 2709.10it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 20%|████████████ | 74/363 [00:00<00:00, 2698.71it/s, Materializing param=model.layers.7.self_attn.v_proj.weight]
|
||
Loading weights: 21%|████████████▍ | 75/363 [00:00<00:00, 2718.54it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 21%|████████████▍ | 75/363 [00:00<00:00, 2708.22it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 21%|████████████▉ | 76/363 [00:00<00:00, 2729.33it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 21%|████████████▉ | 76/363 [00:00<00:00, 2718.77it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 21%|█████████████▏ | 77/363 [00:00<00:00, 2703.67it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 21%|█████████████▏ | 77/363 [00:00<00:00, 2692.71it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 21%|█████████████▊ | 78/363 [00:00<00:00, 2712.67it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 21%|█████████████▊ | 78/363 [00:00<00:00, 2702.69it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 22%|███████████ | 79/363 [00:00<00:00, 2722.41it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 22%|███████████ | 79/363 [00:00<00:00, 2712.18it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 22%|█████████████ | 80/363 [00:00<00:00, 2731.62it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 22%|█████████████ | 80/363 [00:00<00:00, 2721.43it/s, Materializing param=model.layers.8.self_attn.k_proj.weight]
|
||
Loading weights: 22%|█████████████▏ | 81/363 [00:00<00:00, 2701.66it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 22%|█████████████▏ | 81/363 [00:00<00:00, 2691.34it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 23%|█████████████▎ | 82/363 [00:00<00:00, 2688.38it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 23%|█████████████▎ | 82/363 [00:00<00:00, 2622.08it/s, Materializing param=model.layers.8.self_attn.q_proj.weight]
|
||
Loading weights: 23%|█████████████▍ | 83/363 [00:00<00:00, 2637.55it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 23%|█████████████▍ | 83/363 [00:00<00:00, 2628.11it/s, Materializing param=model.layers.8.self_attn.v_proj.weight]
|
||
Loading weights: 23%|█████████████▉ | 84/363 [00:00<00:00, 2646.07it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 23%|█████████████▉ | 84/363 [00:00<00:00, 2637.22it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 23%|██████████████▌ | 85/363 [00:00<00:00, 2655.37it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 23%|██████████████▌ | 85/363 [00:00<00:00, 2646.58it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 24%|██████████████▋ | 86/363 [00:00<00:00, 2664.70it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██████████████▋ | 86/363 [00:00<00:00, 2656.09it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 24%|███████████████▎ | 87/363 [00:00<00:00, 2655.47it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 24%|███████████████▎ | 87/363 [00:00<00:00, 2646.52it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 24%|████████████▎ | 88/363 [00:00<00:00, 2579.49it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 24%|████████████▎ | 88/363 [00:00<00:00, 2570.92it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██████████████▍ | 89/363 [00:00<00:00, 2588.09it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 25%|██████████████▍ | 89/363 [00:00<00:00, 2579.49it/s, Materializing param=model.layers.9.self_attn.k_proj.weight]
|
||
Loading weights: 25%|██████████████▋ | 90/363 [00:00<00:00, 2596.31it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 25%|██████████████▋ | 90/363 [00:00<00:00, 2588.10it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 25%|██████████████▊ | 91/363 [00:00<00:00, 2577.05it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 25%|██████████████▊ | 91/363 [00:00<00:00, 2568.29it/s, Materializing param=model.layers.9.self_attn.q_proj.weight]
|
||
Loading weights: 25%|██████████████▉ | 92/363 [00:00<00:00, 2584.34it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 25%|██████████████▉ | 92/363 [00:00<00:00, 2571.51it/s, Materializing param=model.layers.9.self_attn.v_proj.weight]
|
||
Loading weights: 26%|███████████████ | 93/363 [00:00<00:00, 2586.90it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 26%|███████████████ | 93/363 [00:00<00:00, 2578.82it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 26%|███████████████▊ | 94/363 [00:00<00:00, 2593.81it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 26%|███████████████▊ | 94/363 [00:00<00:00, 2585.95it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 26%|███████████████▉ | 95/363 [00:00<00:00, 2545.66it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 26%|███████████████▉ | 95/363 [00:00<00:00, 2536.06it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 26%|████████████████▋ | 96/363 [00:00<00:00, 2550.63it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 26%|████████████████▋ | 96/363 [00:00<00:00, 2542.39it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 27%|█████████████▎ | 97/363 [00:00<00:00, 2554.84it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 27%|█████████████▎ | 97/363 [00:00<00:00, 2541.67it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 27%|███████████████▋ | 98/363 [00:00<00:00, 2551.82it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 27%|███████████████▋ | 98/363 [00:00<00:00, 2542.51it/s, Materializing param=model.layers.10.self_attn.k_proj.weight]
|
||
Loading weights: 27%|███████████████▊ | 99/363 [00:00<00:00, 2518.49it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 27%|███████████████▊ | 99/363 [00:00<00:00, 2509.25it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 28%|███████████████▋ | 100/363 [00:00<00:00, 2523.04it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 28%|███████████████▋ | 100/363 [00:00<00:00, 2515.63it/s, Materializing param=model.layers.10.self_attn.q_proj.weight]
|
||
Loading weights: 28%|███████████████▊ | 101/363 [00:00<00:00, 2529.34it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 28%|███████████████▊ | 101/363 [00:00<00:00, 2522.19it/s, Materializing param=model.layers.10.self_attn.v_proj.weight]
|
||
Loading weights: 28%|████████████████▎ | 102/363 [00:00<00:00, 2515.13it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 28%|████████████████▎ | 102/363 [00:00<00:00, 2488.43it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 28%|█████████████████ | 103/363 [00:00<00:00, 2501.84it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 28%|█████████████████ | 103/363 [00:00<00:00, 2495.10it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 29%|█████████████████▏ | 104/363 [00:00<00:00, 2507.81it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 29%|█████████████████▏ | 104/363 [00:00<00:00, 2500.87it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 29%|█████████████████▉ | 105/363 [00:00<00:00, 2514.04it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 29%|█████████████████▉ | 105/363 [00:00<00:00, 2506.70it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 29%|██████████████▎ | 106/363 [00:00<00:00, 2475.76it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██████████████▎ | 106/363 [00:00<00:00, 2466.79it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 29%|████████████████▊ | 107/363 [00:00<00:00, 2478.86it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 29%|████████████████▊ | 107/363 [00:00<00:00, 2471.87it/s, Materializing param=model.layers.11.self_attn.k_proj.weight]
|
||
Loading weights: 30%|████████████████▉ | 108/363 [00:00<00:00, 2483.29it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 30%|████████████████▉ | 108/363 [00:00<00:00, 2476.50it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 30%|█████████████████ | 109/363 [00:00<00:00, 2451.65it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 30%|█████████████████ | 109/363 [00:00<00:00, 2445.24it/s, Materializing param=model.layers.11.self_attn.q_proj.weight]
|
||
Loading weights: 30%|█████████████████▎ | 110/363 [00:00<00:00, 2457.46it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 30%|█████████████████▎ | 110/363 [00:00<00:00, 2451.12it/s, Materializing param=model.layers.11.self_attn.v_proj.weight]
|
||
Loading weights: 31%|█████████████████▋ | 111/363 [00:00<00:00, 2463.63it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 31%|█████████████████▋ | 111/363 [00:00<00:00, 2457.12it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 31%|██████████████████▌ | 112/363 [00:00<00:00, 2432.95it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 31%|██████████████████▌ | 112/363 [00:00<00:00, 2423.23it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 31%|██████████████████▋ | 113/363 [00:00<00:00, 2435.17it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 31%|██████████████████▋ | 113/363 [00:00<00:00, 2427.83it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███████████████████▍ | 114/363 [00:00<00:00, 2440.27it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 31%|███████████████████▍ | 114/363 [00:00<00:00, 2434.06it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 32%|███████████████▌ | 115/363 [00:00<00:00, 2421.75it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███████████████▌ | 115/363 [00:00<00:00, 2415.15it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 32%|██████████████████▏ | 116/363 [00:00<00:00, 2426.94it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 32%|██████████████████▏ | 116/363 [00:00<00:00, 2421.03it/s, Materializing param=model.layers.12.self_attn.k_proj.weight]
|
||
Loading weights: 32%|██████████████████▎ | 117/363 [00:00<00:00, 2431.25it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 32%|██████████████████▎ | 117/363 [00:00<00:00, 2422.81it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 33%|██████████████████▌ | 118/363 [00:00<00:00, 2355.76it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 33%|██████████████████▌ | 118/363 [00:00<00:00, 2333.57it/s, Materializing param=model.layers.12.self_attn.q_proj.weight]
|
||
Loading weights: 33%|██████████████████▋ | 119/363 [00:00<00:00, 2339.09it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 33%|██████████████████▋ | 119/363 [00:00<00:00, 2333.60it/s, Materializing param=model.layers.12.self_attn.v_proj.weight]
|
||
Loading weights: 33%|███████████████████▏ | 120/363 [00:00<00:00, 2344.31it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 33%|███████████████████▏ | 120/363 [00:00<00:00, 2330.05it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 33%|████████████████████ | 121/363 [00:00<00:00, 2338.87it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 33%|████████████████████ | 121/363 [00:00<00:00, 2313.48it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 34%|████████████████████▏ | 122/363 [00:00<00:00, 2314.87it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 34%|████████████████████▏ | 122/363 [00:00<00:00, 2309.14it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 34%|█████████████████████ | 123/363 [00:00<00:00, 2319.62it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 34%|█████████████████████ | 123/363 [00:00<00:00, 2314.03it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 34%|████████████████▋ | 124/363 [00:00<00:00, 2324.95it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 34%|████████████████▋ | 124/363 [00:00<00:00, 2318.73it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 34%|███████████████████▋ | 125/363 [00:00<00:00, 2329.12it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 34%|███████████████████▋ | 125/363 [00:00<00:00, 2323.99it/s, Materializing param=model.layers.13.self_attn.k_proj.weight]
|
||
Loading weights: 35%|███████████████████▊ | 126/363 [00:00<00:00, 2334.45it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 35%|███████████████████▊ | 126/363 [00:00<00:00, 2329.33it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 35%|███████████████████▉ | 127/363 [00:00<00:00, 2339.53it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 35%|███████████████████▉ | 127/363 [00:00<00:00, 2334.34it/s, Materializing param=model.layers.13.self_attn.q_proj.weight]
|
||
Loading weights: 35%|████████████████████ | 128/363 [00:00<00:00, 2344.17it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 35%|████████████████████ | 128/363 [00:00<00:00, 2338.79it/s, Materializing param=model.layers.13.self_attn.v_proj.weight]
|
||
Loading weights: 36%|████████████████████▌ | 129/363 [00:00<00:00, 2348.81it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 36%|████████████████████▌ | 129/363 [00:00<00:00, 2343.72it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 36%|█████████████████████▍ | 130/363 [00:00<00:00, 2354.32it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 36%|█████████████████████▍ | 130/363 [00:00<00:00, 2348.72it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 36%|█████████████████████▋ | 131/363 [00:00<00:00, 2358.11it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 36%|█████████████████████▋ | 131/363 [00:00<00:00, 2352.82it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 36%|██████████████████████▌ | 132/363 [00:00<00:00, 2362.65it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 36%|██████████████████████▌ | 132/363 [00:00<00:00, 2355.59it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 37%|█████████████████▉ | 133/363 [00:00<00:00, 2352.63it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 37%|█████████████████▉ | 133/363 [00:00<00:00, 2344.30it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 37%|█████████████████████ | 134/363 [00:00<00:00, 2339.77it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 37%|█████████████████████ | 134/363 [00:00<00:00, 2334.45it/s, Materializing param=model.layers.14.self_attn.k_proj.weight]
|
||
Loading weights: 37%|█████████████████████▏ | 135/363 [00:00<00:00, 2337.36it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 37%|█████████████████████▏ | 135/363 [00:00<00:00, 2330.02it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 37%|█████████████████████▎ | 136/363 [00:00<00:00, 2338.97it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 37%|█████████████████████▎ | 136/363 [00:00<00:00, 2332.58it/s, Materializing param=model.layers.14.self_attn.q_proj.weight]
|
||
Loading weights: 38%|█████████████████████▌ | 137/363 [00:00<00:00, 2325.07it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 38%|█████████████████████▌ | 137/363 [00:00<00:00, 2319.43it/s, Materializing param=model.layers.14.self_attn.v_proj.weight]
|
||
Loading weights: 38%|██████████████████████ | 138/363 [00:00<00:00, 2329.11it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 38%|██████████████████████ | 138/363 [00:00<00:00, 2324.36it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 38%|██████████████████████▉ | 139/363 [00:00<00:00, 2326.19it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 38%|██████████████████████▉ | 139/363 [00:00<00:00, 2321.30it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 39%|███████████████████████▏ | 140/363 [00:00<00:00, 2328.88it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███████████████████████▏ | 140/363 [00:00<00:00, 2317.12it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 39%|████████████████████████ | 141/363 [00:00<00:00, 2307.38it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 39%|████████████████████████ | 141/363 [00:00<00:00, 2302.64it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 39%|███████████████████▏ | 142/363 [00:00<00:00, 2311.88it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 39%|███████████████████▏ | 142/363 [00:00<00:00, 2306.45it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 39%|██████████████████████▍ | 143/363 [00:00<00:00, 2304.73it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 39%|██████████████████████▍ | 143/363 [00:00<00:00, 2299.59it/s, Materializing param=model.layers.15.self_attn.k_proj.weight]
|
||
Loading weights: 40%|██████████████████████▌ | 144/363 [00:00<00:00, 2301.91it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 40%|██████████████████████▌ | 144/363 [00:00<00:00, 2296.72it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 40%|██████████████████████▊ | 145/363 [00:00<00:00, 2304.21it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 40%|██████████████████████▊ | 145/363 [00:00<00:00, 2295.23it/s, Materializing param=model.layers.15.self_attn.q_proj.weight]
|
||
Loading weights: 40%|██████████████████████▉ | 146/363 [00:00<00:00, 2300.95it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 40%|██████████████████████▉ | 146/363 [00:00<00:00, 2293.88it/s, Materializing param=model.layers.15.self_attn.v_proj.weight]
|
||
Loading weights: 40%|███████████████████████▍ | 147/363 [00:00<00:00, 2283.40it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 40%|███████████████████████▍ | 147/363 [00:00<00:00, 2269.04it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 41%|████████████████████████▍ | 148/363 [00:00<00:00, 2277.69it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 41%|████████████████████████▍ | 148/363 [00:00<00:00, 2265.02it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 41%|████████████████████████▋ | 149/363 [00:00<00:00, 2270.49it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 41%|████████████████████████▋ | 149/363 [00:00<00:00, 2260.58it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 41%|█████████████████████████▌ | 150/363 [00:00<00:00, 2269.25it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 41%|█████████████████████████▌ | 150/363 [00:00<00:00, 2264.13it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 42%|████████████████████▍ | 151/363 [00:00<00:00, 2264.13it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 42%|████████████████████▍ | 151/363 [00:00<00:00, 2259.97it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 42%|███████████████████████▊ | 152/363 [00:00<00:00, 2268.66it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 42%|███████████████████████▊ | 152/363 [00:00<00:00, 2262.84it/s, Materializing param=model.layers.16.self_attn.k_proj.weight]
|
||
Loading weights: 42%|████████████████████████ | 153/363 [00:00<00:00, 2271.57it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 42%|████████████████████████ | 153/363 [00:00<00:00, 2265.24it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 42%|████████████████████████▏ | 154/363 [00:00<00:00, 2273.39it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 42%|████████████████████████▏ | 154/363 [00:00<00:00, 2255.96it/s, Materializing param=model.layers.16.self_attn.q_proj.weight]
|
||
Loading weights: 43%|████████████████████████▎ | 155/363 [00:00<00:00, 2254.12it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 43%|████████████████████████▎ | 155/363 [00:00<00:00, 2246.72it/s, Materializing param=model.layers.16.self_attn.v_proj.weight]
|
||
Loading weights: 43%|████████████████████████▉ | 156/363 [00:00<00:00, 2244.22it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 43%|████████████████████████▉ | 156/363 [00:00<00:00, 2238.32it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 43%|█████████████████████████▉ | 157/363 [00:00<00:00, 2246.64it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 43%|█████████████████████████▉ | 157/363 [00:00<00:00, 2236.27it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 44%|██████████████████████████ | 158/363 [00:00<00:00, 2244.55it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 44%|██████████████████████████ | 158/363 [00:00<00:00, 2233.64it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 44%|███████████████████████████▏ | 159/363 [00:00<00:00, 2241.38it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 44%|███████████████████████████▏ | 159/363 [00:00<00:00, 2237.30it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 44%|█████████████████████▌ | 160/363 [00:00<00:00, 2245.65it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 44%|█████████████████████▌ | 160/363 [00:00<00:00, 2233.31it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 44%|█████████████████████████▎ | 161/363 [00:00<00:00, 2238.82it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 44%|█████████████████████████▎ | 161/363 [00:00<00:00, 2231.98it/s, Materializing param=model.layers.17.self_attn.k_proj.weight]
|
||
Loading weights: 45%|█████████████████████████▍ | 162/363 [00:00<00:00, 2237.30it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 45%|█████████████████████████▍ | 162/363 [00:00<00:00, 2224.11it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 45%|█████████████████████████▌ | 163/363 [00:00<00:00, 2230.71it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 45%|█████████████████████████▌ | 163/363 [00:00<00:00, 2224.93it/s, Materializing param=model.layers.17.self_attn.q_proj.weight]
|
||
Loading weights: 45%|█████████████████████████▊ | 164/363 [00:00<00:00, 2230.51it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 45%|█████████████████████████▊ | 164/363 [00:00<00:00, 2226.78it/s, Materializing param=model.layers.17.self_attn.v_proj.weight]
|
||
Loading weights: 45%|██████████████████████████▎ | 165/363 [00:00<00:00, 2235.00it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 45%|██████████████████████████▎ | 165/363 [00:00<00:00, 2229.87it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 46%|███████████████████████████▍ | 166/363 [00:00<00:00, 2238.00it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 46%|███████████████████████████▍ | 166/363 [00:00<00:00, 2234.43it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 46%|███████████████████████████▌ | 167/363 [00:00<00:00, 2238.58it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 46%|███████████████████████████▌ | 167/363 [00:00<00:00, 2234.75it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 46%|████████████████████████████▋ | 168/363 [00:00<00:00, 2242.74it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 46%|████████████████████████████▋ | 168/363 [00:00<00:00, 2239.14it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 47%|██████████████████████▊ | 169/363 [00:00<00:00, 2247.08it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 47%|██████████████████████▊ | 169/363 [00:00<00:00, 2243.43it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 47%|██████████████████████████▋ | 170/363 [00:00<00:00, 2250.82it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 47%|██████████████████████████▋ | 170/363 [00:00<00:00, 2247.24it/s, Materializing param=model.layers.18.self_attn.k_proj.weight]
|
||
Loading weights: 47%|██████████████████████████▊ | 171/363 [00:00<00:00, 2255.19it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 47%|██████████████████████████▊ | 171/363 [00:00<00:00, 2251.16it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 47%|███████████████████████████ | 172/363 [00:00<00:00, 2216.67it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 47%|███████████████████████████ | 172/363 [00:00<00:00, 2213.01it/s, Materializing param=model.layers.18.self_attn.q_proj.weight]
|
||
Loading weights: 48%|███████████████████████████▏ | 173/363 [00:00<00:00, 2220.91it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 48%|███████████████████████████▏ | 173/363 [00:00<00:00, 2217.66it/s, Materializing param=model.layers.18.self_attn.v_proj.weight]
|
||
Loading weights: 48%|███████████████████████████▊ | 174/363 [00:00<00:00, 2225.64it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 48%|███████████████████████████▊ | 174/363 [00:00<00:00, 2222.42it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 48%|████████████████████████████▉ | 175/363 [00:00<00:00, 2230.10it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 48%|████████████████████████████▉ | 175/363 [00:00<00:00, 2226.74it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 48%|█████████████████████████████ | 176/363 [00:00<00:00, 2234.29it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 48%|█████████████████████████████ | 176/363 [00:00<00:00, 2231.05it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 49%|██████████████████████████████▏ | 177/363 [00:00<00:00, 2238.95it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 49%|██████████████████████████████▏ | 177/363 [00:00<00:00, 2235.78it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 49%|████████████████████████ | 178/363 [00:00<00:00, 2243.61it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 49%|████████████████████████ | 178/363 [00:00<00:00, 2240.02it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 49%|████████████████████████████ | 179/363 [00:00<00:00, 2247.54it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 49%|████████████████████████████ | 179/363 [00:00<00:00, 2244.29it/s, Materializing param=model.layers.19.self_attn.k_proj.weight]
|
||
Loading weights: 50%|████████████████████████████▎ | 180/363 [00:00<00:00, 2252.08it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 50%|████████████████████████████▎ | 180/363 [00:00<00:00, 2248.84it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 50%|████████████████████████████▍ | 181/363 [00:00<00:00, 2256.54it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 50%|████████████████████████████▍ | 181/363 [00:00<00:00, 2253.32it/s, Materializing param=model.layers.19.self_attn.q_proj.weight]
|
||
Loading weights: 50%|████████████████████████████▌ | 182/363 [00:00<00:00, 2246.59it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 50%|████████████████████████████▌ | 182/363 [00:00<00:00, 2243.17it/s, Materializing param=model.layers.19.self_attn.v_proj.weight]
|
||
Loading weights: 50%|█████████████████████████████▏ | 183/363 [00:00<00:00, 2250.72it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 50%|█████████████████████████████▏ | 183/363 [00:00<00:00, 2247.18it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 51%|██████████████████████████████▍ | 184/363 [00:00<00:00, 2254.51it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 51%|██████████████████████████████▍ | 184/363 [00:00<00:00, 2251.27it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 51%|██████████████████████████████▌ | 185/363 [00:00<00:00, 2254.89it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 51%|██████████████████████████████▌ | 185/363 [00:00<00:00, 2251.52it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 51%|███████████████████████████████▊ | 186/363 [00:00<00:00, 2246.86it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 51%|███████████████████████████████▊ | 186/363 [00:00<00:00, 2238.10it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 52%|█████████████████████████▏ | 187/363 [00:00<00:00, 2245.24it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 52%|█████████████████████████▏ | 187/363 [00:00<00:00, 2242.05it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 52%|█████████████████████████████▌ | 188/363 [00:00<00:00, 2248.91it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 52%|█████████████████████████████▌ | 188/363 [00:00<00:00, 2245.72it/s, Materializing param=model.layers.20.self_attn.k_proj.weight]
|
||
Loading weights: 52%|█████████████████████████████▋ | 189/363 [00:00<00:00, 2252.90it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 52%|█████████████████████████████▋ | 189/363 [00:00<00:00, 2249.74it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 52%|█████████████████████████████▊ | 190/363 [00:00<00:00, 2244.62it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 52%|█████████████████████████████▊ | 190/363 [00:00<00:00, 2241.26it/s, Materializing param=model.layers.20.self_attn.q_proj.weight]
|
||
Loading weights: 53%|█████████████████████████████▉ | 191/363 [00:00<00:00, 2248.20it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 53%|█████████████████████████████▉ | 191/363 [00:00<00:00, 2245.02it/s, Materializing param=model.layers.20.self_attn.v_proj.weight]
|
||
Loading weights: 53%|██████████████████████████████▋ | 192/363 [00:00<00:00, 2238.95it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 53%|██████████████████████████████▋ | 192/363 [00:00<00:00, 2235.30it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 53%|███████████████████████████████▉ | 193/363 [00:00<00:00, 2242.12it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 53%|███████████████████████████████▉ | 193/363 [00:00<00:00, 2238.87it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 53%|████████████████████████████████ | 194/363 [00:00<00:00, 2228.58it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 53%|████████████████████████████████ | 194/363 [00:00<00:00, 2225.27it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 54%|█████████████████████████████████▎ | 195/363 [00:00<00:00, 2228.23it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 54%|█████████████████████████████████▎ | 195/363 [00:00<00:00, 2225.13it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 54%|██████████████████████████▍ | 196/363 [00:00<00:00, 2220.50it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 54%|██████████████████████████▍ | 196/363 [00:00<00:00, 2217.42it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 54%|██████████████████████████████▉ | 197/363 [00:00<00:00, 2223.89it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 54%|██████████████████████████████▉ | 197/363 [00:00<00:00, 2220.86it/s, Materializing param=model.layers.21.self_attn.k_proj.weight]
|
||
Loading weights: 55%|███████████████████████████████ | 198/363 [00:00<00:00, 2227.63it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 55%|███████████████████████████████ | 198/363 [00:00<00:00, 2224.62it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 55%|███████████████████████████████▏ | 199/363 [00:00<00:00, 2231.43it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 55%|███████████████████████████████▏ | 199/363 [00:00<00:00, 2228.48it/s, Materializing param=model.layers.21.self_attn.q_proj.weight]
|
||
Loading weights: 55%|███████████████████████████████▍ | 200/363 [00:00<00:00, 2235.06it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 55%|███████████████████████████████▍ | 200/363 [00:00<00:00, 2231.96it/s, Materializing param=model.layers.21.self_attn.v_proj.weight]
|
||
Loading weights: 55%|████████████████████████████████ | 201/363 [00:00<00:00, 2218.95it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 55%|████████████████████████████████ | 201/363 [00:00<00:00, 2215.40it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 56%|█████████████████████████████████▍ | 202/363 [00:00<00:00, 2221.83it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████████████████████████████████▍ | 202/363 [00:00<00:00, 2218.94it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████████████████████████████████▌ | 203/363 [00:00<00:00, 2210.92it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████████████████████████████████▌ | 203/363 [00:00<00:00, 2207.87it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 56%|██████████████████████████████████▊ | 204/363 [00:00<00:00, 2214.38it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 56%|██████████████████████████████████▊ | 204/363 [00:00<00:00, 2211.53it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 56%|███████████████████████████▋ | 205/363 [00:00<00:00, 2218.11it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 56%|███████████████████████████▋ | 205/363 [00:00<00:00, 2214.96it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 57%|████████████████████████████████▎ | 206/363 [00:00<00:00, 2208.39it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 57%|████████████████████████████████▎ | 206/363 [00:00<00:00, 2205.48it/s, Materializing param=model.layers.22.self_attn.k_proj.weight]
|
||
Loading weights: 57%|████████████████████████████████▌ | 207/363 [00:00<00:00, 2211.11it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 57%|████████████████████████████████▌ | 207/363 [00:00<00:00, 2208.25it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 57%|████████████████████████████████▋ | 208/363 [00:00<00:00, 2214.81it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 57%|████████████████████████████████▋ | 208/363 [00:00<00:00, 2212.05it/s, Materializing param=model.layers.22.self_attn.q_proj.weight]
|
||
Loading weights: 58%|████████████████████████████████▊ | 209/363 [00:00<00:00, 2218.58it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 58%|████████████████████████████████▊ | 209/363 [00:00<00:00, 2215.80it/s, Materializing param=model.layers.22.self_attn.v_proj.weight]
|
||
Loading weights: 58%|█████████████████████████████████▌ | 210/363 [00:00<00:00, 2217.62it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 58%|█████████████████████████████████▌ | 210/363 [00:00<00:00, 2214.60it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 58%|██████████████████████████████████▉ | 211/363 [00:00<00:00, 2189.96it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 58%|██████████████████████████████████▉ | 211/363 [00:00<00:00, 2186.74it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 58%|███████████████████████████████████ | 212/363 [00:00<00:00, 2192.86it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 58%|███████████████████████████████████ | 212/363 [00:00<00:00, 2190.11it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 59%|████████████████████████████████████▍ | 213/363 [00:00<00:00, 2196.43it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 59%|████████████████████████████████████▍ | 213/363 [00:00<00:00, 2193.75it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 59%|████████████████████████████▉ | 214/363 [00:00<00:00, 2200.03it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 59%|████████████████████████████▉ | 214/363 [00:00<00:00, 2197.27it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████████████████████████████████▊ | 215/363 [00:00<00:00, 2202.73it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 59%|█████████████████████████████████▊ | 215/363 [00:00<00:00, 2200.00it/s, Materializing param=model.layers.23.self_attn.k_proj.weight]
|
||
Loading weights: 60%|█████████████████████████████████▉ | 216/363 [00:00<00:00, 2206.24it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 60%|█████████████████████████████████▉ | 216/363 [00:00<00:00, 2203.57it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 60%|██████████████████████████████████ | 217/363 [00:00<00:00, 2209.76it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 60%|██████████████████████████████████ | 217/363 [00:00<00:00, 2207.05it/s, Materializing param=model.layers.23.self_attn.q_proj.weight]
|
||
Loading weights: 60%|██████████████████████████████████▏ | 218/363 [00:00<00:00, 2198.17it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 60%|██████████████████████████████████▏ | 218/363 [00:00<00:00, 2195.37it/s, Materializing param=model.layers.23.self_attn.v_proj.weight]
|
||
Loading weights: 60%|██████████████████████████████████▉ | 219/363 [00:00<00:00, 2201.17it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 60%|██████████████████████████████████▉ | 219/363 [00:00<00:00, 2198.40it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 61%|███████████████████████████████████▏ | 220/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 61%|████████████████████████████████████▎ | 220/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 61%|████████████████████████████████████▎ | 220/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 61%|████████████████████████████████████▌ | 221/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 61%|████████████████████████████████████▌ | 221/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 61%|█████████████████████████████████████▉ | 222/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 61%|█████████████████████████████████████▉ | 222/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 61%|██████████████████████████████ | 223/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 61%|██████████████████████████████ | 223/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 62%|███████████████████████████████████▏ | 224/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 62%|███████████████████████████████████▏ | 224/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.k_proj.weight]
|
||
Loading weights: 62%|███████████████████████████████████▎ | 225/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 62%|███████████████████████████████████▎ | 225/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 62%|███████████████████████████████████▍ | 226/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 62%|███████████████████████████████████▍ | 226/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.q_proj.weight]
|
||
Loading weights: 63%|███████████████████████████████████▋ | 227/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 63%|███████████████████████████████████▋ | 227/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.24.self_attn.v_proj.weight]
|
||
Loading weights: 63%|████████████████████████████████████▍ | 228/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 63%|████████████████████████████████████▍ | 228/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 63%|█████████████████████████████████████▊ | 229/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 63%|█████████████████████████████████████▊ | 229/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████████████████████████████████████ | 230/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████████████████████████████████████ | 230/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 64%|███████████████████████████████████████▍ | 231/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 64%|███████████████████████████████████████▍ | 231/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 64%|███████████████████████████████▎ | 232/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 64%|███████████████████████████████▎ | 232/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 64%|████████████████████████████████████▌ | 233/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 64%|████████████████████████████████████▌ | 233/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.k_proj.weight]
|
||
Loading weights: 64%|████████████████████████████████████▋ | 234/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 64%|████████████████████████████████████▋ | 234/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 65%|████████████████████████████████████▉ | 235/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 65%|████████████████████████████████████▉ | 235/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.q_proj.weight]
|
||
Loading weights: 65%|█████████████████████████████████████ | 236/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 65%|█████████████████████████████████████ | 236/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.25.self_attn.v_proj.weight]
|
||
Loading weights: 65%|█████████████████████████████████████▊ | 237/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 65%|█████████████████████████████████████▊ | 237/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 66%|███████████████████████████████████████▎ | 238/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 66%|███████████████████████████████████████▎ | 238/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 66%|███████████████████████████████████████▌ | 239/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 66%|███████████████████████████████████████▌ | 239/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 66%|████████████████████████████████████████▉ | 240/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 66%|████████████████████████████████████████▉ | 240/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 66%|████████████████████████████████▌ | 241/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 66%|████████████████████████████████▌ | 241/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████████████████████████████████████ | 242/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 67%|██████████████████████████████████████ | 242/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.k_proj.weight]
|
||
Loading weights: 67%|██████████████████████████████████████▏ | 243/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 67%|██████████████████████████████████████▏ | 243/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 67%|██████████████████████████████████████▎ | 244/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 67%|██████████████████████████████████████▎ | 244/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.q_proj.weight]
|
||
Loading weights: 67%|██████████████████████████████████████▍ | 245/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 67%|██████████████████████████████████████▍ | 245/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.26.self_attn.v_proj.weight]
|
||
Loading weights: 68%|███████████████████████████████████████▎ | 246/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 68%|███████████████████████████████████████▎ | 246/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 68%|████████████████████████████████████████▊ | 247/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 68%|████████████████████████████████████████▊ | 247/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 68%|████████████████████████████████████████▉ | 248/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 68%|████████████████████████████████████████▉ | 248/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 69%|██████████████████████████████████████████▌ | 249/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 69%|██████████████████████████████████████████▌ | 249/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 69%|█████████████████████████████████▋ | 250/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 69%|█████████████████████████████████▋ | 250/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 69%|███████████████████████████████████████▍ | 251/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 69%|███████████████████████████████████████▍ | 251/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.k_proj.weight]
|
||
Loading weights: 69%|███████████████████████████████████████▌ | 252/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 69%|███████████████████████████████████████▌ | 252/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 70%|███████████████████████████████████████▋ | 253/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 70%|███████████████████████████████████████▋ | 253/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.q_proj.weight]
|
||
Loading weights: 70%|███████████████████████████████████████▉ | 254/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 70%|███████████████████████████████████████▉ | 254/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.27.self_attn.v_proj.weight]
|
||
Loading weights: 70%|████████████████████████████████████████▋ | 255/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.input_layernorm.weight]
|
||
Loading weights: 70%|████████████████████████████████████████▋ | 255/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.input_layernorm.weight]
|
||
Loading weights: 71%|██████████████████████████████████████████▎ | 256/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
|
||
Loading weights: 71%|██████████████████████████████████████████▎ | 256/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.down_proj.weight]
|
||
Loading weights: 71%|██████████████████████████████████████████▍ | 257/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
|
||
Loading weights: 71%|██████████████████████████████████████████▍ | 257/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.gate_proj.weight]
|
||
Loading weights: 71%|████████████████████████████████████████████ | 258/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
|
||
Loading weights: 71%|████████████████████████████████████████████ | 258/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.mlp.up_proj.weight]
|
||
Loading weights: 71%|██████████████████████████████████▉ | 259/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
|
||
Loading weights: 71%|██████████████████████████████████▉ | 259/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.post_attention_layernorm.weight]
|
||
Loading weights: 72%|████████████████████████████████████████▊ | 260/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
|
||
Loading weights: 72%|████████████████████████████████████████▊ | 260/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.k_proj.weight]
|
||
Loading weights: 72%|████████████████████████████████████████▉ | 261/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
|
||
Loading weights: 72%|████████████████████████████████████████▉ | 261/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.o_proj.weight]
|
||
Loading weights: 72%|█████████████████████████████████████████▏ | 262/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
|
||
Loading weights: 72%|█████████████████████████████████████████▏ | 262/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.q_proj.weight]
|
||
Loading weights: 72%|█████████████████████████████████████████▎ | 263/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
|
||
Loading weights: 72%|█████████████████████████████████████████▎ | 263/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.28.self_attn.v_proj.weight]
|
||
Loading weights: 73%|██████████████████████████████████████████▏ | 264/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.input_layernorm.weight]
|
||
Loading weights: 73%|██████████████████████████████████████████▏ | 264/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.input_layernorm.weight]
|
||
Loading weights: 73%|███████████████████████████████████████████▊ | 265/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████████████████████████████████████████▊ | 265/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████████████████████████████████████████▉ | 266/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████████████████████████████████████████▉ | 266/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.gate_proj.weight]
|
||
Loading weights: 74%|█████████████████████████████████████████████▌ | 267/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
|
||
Loading weights: 74%|█████████████████████████████████████████████▌ | 267/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.mlp.up_proj.weight]
|
||
Loading weights: 74%|████████████████████████████████████▏ | 268/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
|
||
Loading weights: 74%|████████████████████████████████████▏ | 268/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.post_attention_layernorm.weight]
|
||
Loading weights: 74%|██████████████████████████████████████████▏ | 269/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
|
||
Loading weights: 74%|██████████████████████████████████████████▏ | 269/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.k_proj.weight]
|
||
Loading weights: 74%|██████████████████████████████████████████▍ | 270/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
|
||
Loading weights: 74%|██████████████████████████████████████████▍ | 270/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.o_proj.weight]
|
||
Loading weights: 75%|██████████████████████████████████████████▌ | 271/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
|
||
Loading weights: 75%|██████████████████████████████████████████▌ | 271/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.q_proj.weight]
|
||
Loading weights: 75%|██████████████████████████████████████████▋ | 272/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
|
||
Loading weights: 75%|██████████████████████████████████████████▋ | 272/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.29.self_attn.v_proj.weight]
|
||
Loading weights: 75%|███████████████████████████████████████████▌ | 273/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.input_layernorm.weight]
|
||
Loading weights: 75%|███████████████████████████████████████████▌ | 273/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.input_layernorm.weight]
|
||
Loading weights: 75%|█████████████████████████████████████████████▎ | 274/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
|
||
Loading weights: 75%|█████████████████████████████████████████████▎ | 274/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.down_proj.weight]
|
||
Loading weights: 76%|█████████████████████████████████████████████▍ | 275/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
|
||
Loading weights: 76%|█████████████████████████████████████████████▍ | 275/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.gate_proj.weight]
|
||
Loading weights: 76%|███████████████████████████████████████████████▏ | 276/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
|
||
Loading weights: 76%|███████████████████████████████████████████████▏ | 276/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.mlp.up_proj.weight]
|
||
Loading weights: 76%|█████████████████████████████████████▍ | 277/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
|
||
Loading weights: 76%|█████████████████████████████████████▍ | 277/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.post_attention_layernorm.weight]
|
||
Loading weights: 77%|███████████████████████████████████████████▋ | 278/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
|
||
Loading weights: 77%|███████████████████████████████████████████▋ | 278/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.k_proj.weight]
|
||
Loading weights: 77%|███████████████████████████████████████████▊ | 279/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
|
||
Loading weights: 77%|███████████████████████████████████████████▊ | 279/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.o_proj.weight]
|
||
Loading weights: 77%|███████████████████████████████████████████▉ | 280/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
|
||
Loading weights: 77%|███████████████████████████████████████████▉ | 280/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.q_proj.weight]
|
||
Loading weights: 77%|████████████████████████████████████████████ | 281/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
|
||
Loading weights: 77%|████████████████████████████████████████████ | 281/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.30.self_attn.v_proj.weight]
|
||
Loading weights: 78%|█████████████████████████████████████████████ | 282/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.input_layernorm.weight]
|
||
Loading weights: 78%|█████████████████████████████████████████████ | 282/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.input_layernorm.weight]
|
||
Loading weights: 78%|██████████████████████████████████████████████▊ | 283/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
|
||
Loading weights: 78%|██████████████████████████████████████████████▊ | 283/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.down_proj.weight]
|
||
Loading weights: 78%|██████████████████████████████████████████████▉ | 284/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
|
||
Loading weights: 78%|██████████████████████████████████████████████▉ | 284/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.gate_proj.weight]
|
||
Loading weights: 79%|████████████████████████████████████████████████▋ | 285/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
|
||
Loading weights: 79%|████████████████████████████████████████████████▋ | 285/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.mlp.up_proj.weight]
|
||
Loading weights: 79%|██████████████████████████████████████▌ | 286/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
|
||
Loading weights: 79%|██████████████████████████████████████▌ | 286/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.post_attention_layernorm.weight]
|
||
Loading weights: 79%|█████████████████████████████████████████████ | 287/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
|
||
Loading weights: 79%|█████████████████████████████████████████████ | 287/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.k_proj.weight]
|
||
Loading weights: 79%|█████████████████████████████████████████████▏ | 288/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
|
||
Loading weights: 79%|█████████████████████████████████████████████▏ | 288/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.o_proj.weight]
|
||
Loading weights: 80%|█████████████████████████████████████████████▍ | 289/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
|
||
Loading weights: 80%|█████████████████████████████████████████████▍ | 289/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.q_proj.weight]
|
||
Loading weights: 80%|█████████████████████████████████████████████▌ | 290/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
|
||
Loading weights: 80%|█████████████████████████████████████████████▌ | 290/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.31.self_attn.v_proj.weight]
|
||
Loading weights: 80%|██████████████████████████████████████████████▍ | 291/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.input_layernorm.weight]
|
||
Loading weights: 80%|██████████████████████████████████████████████▍ | 291/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.input_layernorm.weight]
|
||
Loading weights: 80%|████████████████████████████████████████████████▎ | 292/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
|
||
Loading weights: 80%|████████████████████████████████████████████████▎ | 292/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.down_proj.weight]
|
||
Loading weights: 81%|████████████████████████████████████████████████▍ | 293/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
|
||
Loading weights: 81%|████████████████████████████████████████████████▍ | 293/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.gate_proj.weight]
|
||
Loading weights: 81%|██████████████████████████████████████████████████▏ | 294/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
|
||
Loading weights: 81%|██████████████████████████████████████████████████▏ | 294/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.mlp.up_proj.weight]
|
||
Loading weights: 81%|███████████████████████████████████████▊ | 295/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
|
||
Loading weights: 81%|███████████████████████████████████████▊ | 295/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.post_attention_layernorm.weight]
|
||
Loading weights: 82%|██████████████████████████████████████████████▍ | 296/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
|
||
Loading weights: 82%|██████████████████████████████████████████████▍ | 296/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.k_proj.weight]
|
||
Loading weights: 82%|██████████████████████████████████████████████▋ | 297/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
|
||
Loading weights: 82%|██████████████████████████████████████████████▋ | 297/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.o_proj.weight]
|
||
Loading weights: 82%|██████████████████████████████████████████████▊ | 298/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
|
||
Loading weights: 82%|██████████████████████████████████████████████▊ | 298/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.q_proj.weight]
|
||
Loading weights: 82%|██████████████████████████████████████████████▉ | 299/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
|
||
Loading weights: 82%|██████████████████████████████████████████████▉ | 299/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.32.self_attn.v_proj.weight]
|
||
Loading weights: 83%|███████████████████████████████████████████████▉ | 300/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.input_layernorm.weight]
|
||
Loading weights: 83%|███████████████████████████████████████████████▉ | 300/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.input_layernorm.weight]
|
||
Loading weights: 83%|█████████████████████████████████████████████████▊ | 301/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
|
||
Loading weights: 83%|█████████████████████████████████████████████████▊ | 301/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.down_proj.weight]
|
||
Loading weights: 83%|█████████████████████████████████████████████████▉ | 302/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
|
||
Loading weights: 83%|█████████████████████████████████████████████████▉ | 302/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.gate_proj.weight]
|
||
Loading weights: 83%|███████████████████████████████████████████████████▊ | 303/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
|
||
Loading weights: 83%|███████████████████████████████████████████████████▊ | 303/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.mlp.up_proj.weight]
|
||
Loading weights: 84%|█████████████████████████████████████████ | 304/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
|
||
Loading weights: 84%|█████████████████████████████████████████ | 304/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.post_attention_layernorm.weight]
|
||
Loading weights: 84%|███████████████████████████████████████████████▉ | 305/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
|
||
Loading weights: 84%|███████████████████████████████████████████████▉ | 305/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.k_proj.weight]
|
||
Loading weights: 84%|████████████████████████████████████████████████ | 306/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
|
||
Loading weights: 84%|████████████████████████████████████████████████ | 306/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████████████████████████████████████████████▏ | 307/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
|
||
Loading weights: 85%|████████████████████████████████████████████████▏ | 307/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.q_proj.weight]
|
||
Loading weights: 85%|████████████████████████████████████████████████▎ | 308/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
|
||
Loading weights: 85%|████████████████████████████████████████████████▎ | 308/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.33.self_attn.v_proj.weight]
|
||
Loading weights: 85%|█████████████████████████████████████████████████▎ | 309/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.input_layernorm.weight]
|
||
Loading weights: 85%|█████████████████████████████████████████████████▎ | 309/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.input_layernorm.weight]
|
||
Loading weights: 85%|███████████████████████████████████████████████████▏ | 310/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
|
||
Loading weights: 85%|███████████████████████████████████████████████████▏ | 310/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.down_proj.weight]
|
||
Loading weights: 86%|███████████████████████████████████████████████████▍ | 311/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
|
||
Loading weights: 86%|███████████████████████████████████████████████████▍ | 311/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.gate_proj.weight]
|
||
Loading weights: 86%|█████████████████████████████████████████████████████▎ | 312/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
|
||
Loading weights: 86%|█████████████████████████████████████████████████████▎ | 312/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.mlp.up_proj.weight]
|
||
Loading weights: 86%|██████████████████████████████████████████▎ | 313/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
|
||
Loading weights: 86%|██████████████████████████████████████████▎ | 313/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.post_attention_layernorm.weight]
|
||
Loading weights: 87%|█████████████████████████████████████████████████▎ | 314/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
|
||
Loading weights: 87%|█████████████████████████████████████████████████▎ | 314/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.k_proj.weight]
|
||
Loading weights: 87%|█████████████████████████████████████████████████▍ | 315/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
|
||
Loading weights: 87%|█████████████████████████████████████████████████▍ | 315/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.o_proj.weight]
|
||
Loading weights: 87%|█████████████████████████████████████████████████▌ | 316/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
|
||
Loading weights: 87%|█████████████████████████████████████████████████▌ | 316/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.q_proj.weight]
|
||
Loading weights: 87%|█████████████████████████████████████████████████▊ | 317/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
|
||
Loading weights: 87%|█████████████████████████████████████████████████▊ | 317/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.34.self_attn.v_proj.weight]
|
||
Loading weights: 88%|██████████████████████████████████████████████████▊ | 318/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.input_layernorm.weight]
|
||
Loading weights: 88%|██████████████████████████████████████████████████▊ | 318/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.input_layernorm.weight]
|
||
Loading weights: 88%|████████████████████████████████████████████████████▋ | 319/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
|
||
Loading weights: 88%|████████████████████████████████████████████████████▋ | 319/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.down_proj.weight]
|
||
Loading weights: 88%|████████████████████████████████████████████████████▉ | 320/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
|
||
Loading weights: 88%|████████████████████████████████████████████████████▉ | 320/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.gate_proj.weight]
|
||
Loading weights: 88%|██████████████████████████████████████████████████████▊ | 321/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
|
||
Loading weights: 88%|██████████████████████████████████████████████████████▊ | 321/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.mlp.up_proj.weight]
|
||
Loading weights: 89%|███████████████████████████████████████████▍ | 322/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
|
||
Loading weights: 89%|███████████████████████████████████████████▍ | 322/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.post_attention_layernorm.weight]
|
||
Loading weights: 89%|██████████████████████████████████████████████████▋ | 323/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
|
||
Loading weights: 89%|██████████████████████████████████████████████████▋ | 323/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.k_proj.weight]
|
||
Loading weights: 89%|██████████████████████████████████████████████████▉ | 324/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
|
||
Loading weights: 89%|██████████████████████████████████████████████████▉ | 324/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.o_proj.weight]
|
||
Loading weights: 90%|███████████████████████████████████████████████████ | 325/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
|
||
Loading weights: 90%|███████████████████████████████████████████████████ | 325/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.q_proj.weight]
|
||
Loading weights: 90%|███████████████████████████████████████████████████▏ | 326/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
|
||
Loading weights: 90%|███████████████████████████████████████████████████▏ | 326/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.35.self_attn.v_proj.weight]
|
||
Loading weights: 90%|████████████████████████████████████████████████████▏ | 327/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.input_layernorm.weight]
|
||
Loading weights: 90%|████████████████████████████████████████████████████▏ | 327/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.input_layernorm.weight]
|
||
Loading weights: 90%|██████████████████████████████████████████████████████▏ | 328/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
|
||
Loading weights: 90%|██████████████████████████████████████████████████████▏ | 328/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.down_proj.weight]
|
||
Loading weights: 91%|██████████████████████████████████████████████████████▍ | 329/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
|
||
Loading weights: 91%|██████████████████████████████████████████████████████▍ | 329/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.gate_proj.weight]
|
||
Loading weights: 91%|████████████████████████████████████████████████████████▎ | 330/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
|
||
Loading weights: 91%|████████████████████████████████████████████████████████▎ | 330/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.mlp.up_proj.weight]
|
||
Loading weights: 91%|████████████████████████████████████████████▋ | 331/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
|
||
Loading weights: 91%|████████████████████████████████████████████▋ | 331/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.post_attention_layernorm.weight]
|
||
Loading weights: 91%|████████████████████████████████████████████████████▏ | 332/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
|
||
Loading weights: 91%|████████████████████████████████████████████████████▏ | 332/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.k_proj.weight]
|
||
Loading weights: 92%|████████████████████████████████████████████████████▎ | 333/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
|
||
Loading weights: 92%|████████████████████████████████████████████████████▎ | 333/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.o_proj.weight]
|
||
Loading weights: 92%|████████████████████████████████████████████████████▍ | 334/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
|
||
Loading weights: 92%|████████████████████████████████████████████████████▍ | 334/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.q_proj.weight]
|
||
Loading weights: 92%|████████████████████████████████████████████████████▌ | 335/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
|
||
Loading weights: 92%|████████████████████████████████████████████████████▌ | 335/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.36.self_attn.v_proj.weight]
|
||
Loading weights: 93%|█████████████████████████████████████████████████████▋ | 336/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.input_layernorm.weight]
|
||
Loading weights: 93%|█████████████████████████████████████████████████████▋ | 336/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.input_layernorm.weight]
|
||
Loading weights: 93%|███████████████████████████████████████████████████████▋ | 337/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
|
||
Loading weights: 93%|███████████████████████████████████████████████████████▋ | 337/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.down_proj.weight]
|
||
Loading weights: 93%|███████████████████████████████████████████████████████▊ | 338/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
|
||
Loading weights: 93%|███████████████████████████████████████████████████████▊ | 338/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.gate_proj.weight]
|
||
Loading weights: 93%|█████████████████████████████████████████████████████████▉ | 339/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
|
||
Loading weights: 93%|█████████████████████████████████████████████████████████▉ | 339/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.mlp.up_proj.weight]
|
||
Loading weights: 94%|█████████████████████████████████████████████▉ | 340/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
|
||
Loading weights: 94%|█████████████████████████████████████████████▉ | 340/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.post_attention_layernorm.weight]
|
||
Loading weights: 94%|█████████████████████████████████████████████████████▌ | 341/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
|
||
Loading weights: 94%|█████████████████████████████████████████████████████▌ | 341/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.k_proj.weight]
|
||
Loading weights: 94%|█████████████████████████████████████████████████████▋ | 342/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
|
||
Loading weights: 94%|█████████████████████████████████████████████████████▋ | 342/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.o_proj.weight]
|
||
Loading weights: 94%|█████████████████████████████████████████████████████▊ | 343/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
|
||
Loading weights: 94%|█████████████████████████████████████████████████████▊ | 343/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.q_proj.weight]
|
||
Loading weights: 95%|██████████████████████████████████████████████████████ | 344/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
|
||
Loading weights: 95%|██████████████████████████████████████████████████████ | 344/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.37.self_attn.v_proj.weight]
|
||
Loading weights: 95%|███████████████████████████████████████████████████████ | 345/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.input_layernorm.weight]
|
||
Loading weights: 95%|███████████████████████████████████████████████████████ | 345/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.input_layernorm.weight]
|
||
Loading weights: 95%|█████████████████████████████████████████████████████████▏ | 346/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
|
||
Loading weights: 95%|█████████████████████████████████████████████████████████▏ | 346/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.down_proj.weight]
|
||
Loading weights: 96%|█████████████████████████████████████████████████████████▎ | 347/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
|
||
Loading weights: 96%|█████████████████████████████████████████████████████████▎ | 347/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.gate_proj.weight]
|
||
Loading weights: 96%|███████████████████████████████████████████████████████████▍ | 348/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
|
||
Loading weights: 96%|███████████████████████████████████████████████████████████▍ | 348/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.mlp.up_proj.weight]
|
||
Loading weights: 96%|███████████████████████████████████████████████ | 349/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
|
||
Loading weights: 96%|███████████████████████████████████████████████ | 349/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.post_attention_layernorm.weight]
|
||
Loading weights: 96%|██████████████████████████████████████████████████████▉ | 350/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
|
||
Loading weights: 96%|██████████████████████████████████████████████████████▉ | 350/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.k_proj.weight]
|
||
Loading weights: 97%|███████████████████████████████████████████████████████ | 351/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
|
||
Loading weights: 97%|███████████████████████████████████████████████████████ | 351/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.o_proj.weight]
|
||
Loading weights: 97%|███████████████████████████████████████████████████████▎ | 352/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
|
||
Loading weights: 97%|███████████████████████████████████████████████████████▎ | 352/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.q_proj.weight]
|
||
Loading weights: 97%|███████████████████████████████████████████████████████▍ | 353/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
|
||
Loading weights: 97%|███████████████████████████████████████████████████████▍ | 353/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.38.self_attn.v_proj.weight]
|
||
Loading weights: 98%|████████████████████████████████████████████████████████▌ | 354/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.input_layernorm.weight]
|
||
Loading weights: 98%|████████████████████████████████████████████████████████▌ | 354/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.input_layernorm.weight]
|
||
Loading weights: 98%|██████████████████████████████████████████████████████████▋ | 355/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
|
||
Loading weights: 98%|██████████████████████████████████████████████████████████▋ | 355/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.down_proj.weight]
|
||
Loading weights: 98%|██████████████████████████████████████████████████████████▊ | 356/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
|
||
Loading weights: 98%|██████████████████████████████████████████████████████████▊ | 356/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.gate_proj.weight]
|
||
Loading weights: 98%|████████████████████████████████████████████████████████████▉ | 357/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
|
||
Loading weights: 98%|████████████████████████████████████████████████████████████▉ | 357/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.mlp.up_proj.weight]
|
||
Loading weights: 99%|████████████████████████████████████████████████▎| 358/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
|
||
Loading weights: 99%|████████████████████████████████████████████████▎| 358/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.post_attention_layernorm.weight]
|
||
Loading weights: 99%|████████████████████████████████████████████████████████▎| 359/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
|
||
Loading weights: 99%|████████████████████████████████████████████████████████▎| 359/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.k_proj.weight]
|
||
Loading weights: 99%|████████████████████████████████████████████████████████▌| 360/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
|
||
Loading weights: 99%|████████████████████████████████████████████████████████▌| 360/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.o_proj.weight]
|
||
Loading weights: 99%|████████████████████████████████████████████████████████▋| 361/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
|
||
Loading weights: 99%|████████████████████████████████████████████████████████▋| 361/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.q_proj.weight]
|
||
Loading weights: 100%|████████████████████████████████████████████████████████▊| 362/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
|
||
Loading weights: 100%|████████████████████████████████████████████████████████▊| 362/363 [00:00<00:00, 2182.81it/s, Materializing param=model.layers.39.self_attn.v_proj.weight]
|
||
Loading weights: 100%|███████████████████████████████████████████████████████████████████████████████| 363/363 [00:00<00:00, 2182.81it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|███████████████████████████████████████████████████████████████████████████████| 363/363 [00:00<00:00, 2182.81it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|███████████████████████████████████████████████████████████████████████████████| 363/363 [00:00<00:00, 2040.45it/s, Materializing param=model.norm.weight]
|
||
[2026-02-06 15:51:29,234] [WARNING] [accelerate.utils.dataclasses.__post_init__:1962] [PID:7207] sync_module_states is obsolete in FSDP2, as it is not needed anymore.Setting sync_module_states to None.Multiple deprecation warnings due to FSDP2 conversion:
|
||
sharding_strategy is deprecated in favor of reshard_after_forward. This will be removed in a future version of Accelerate.
|
||
[2026-02-06 15:51:34,875] [WARNING] [py.warnings._showwarnmsg:110] [PID:7207] /root/axolotl/.venv/lib/python3.11/site-packages/torch/distributed/distributed_c10d.py:4807: UserWarning: No device id is provided via `init_process_group` or `barrier `. Using the current device set by the user.
|
||
warnings.warn( # warn only once
|
||
|
||
[0m |