1194 lines
554 KiB
Plaintext
1194 lines
554 KiB
Plaintext
2026-03-05 03:23:38 [INFO] ========================================================================
|
||
2026-03-05 03:23:38 [INFO] FRANKENSTALLM 3B — Full Evaluation Pipeline
|
||
2026-03-05 03:23:38 [INFO] ========================================================================
|
||
2026-03-05 03:23:38 [INFO] Checkpoint : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/checkpoints/korean_3b_fp8_run1/checkpoint-0057000
|
||
2026-03-05 03:23:38 [INFO] Tokenizer : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/tokenizer/korean_sp/tokenizer.json
|
||
2026-03-05 03:23:38 [INFO] Data dir : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/data
|
||
2026-03-05 03:23:38 [INFO] Output dir : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0323
|
||
2026-03-05 03:23:38 [INFO] GPUs : [2, 3, 4, 5, 6, 7]
|
||
2026-03-05 03:23:38 [INFO] SEQ_LEN : 2048 STRIDE: 512 BATCH_SIZE: 32
|
||
2026-03-05 03:23:38 [INFO] Phases : phase0=skip phase1=run phase2=run
|
||
2026-03-05 03:23:38 [INFO]
|
||
2026-03-05 03:23:38 [INFO] ------------------------------------------------------------------------
|
||
2026-03-05 03:23:38 [INFO] PHASE 0 — HF Checkpoint Conversion
|
||
2026-03-05 03:23:38 [INFO] ------------------------------------------------------------------------
|
||
2026-03-05 03:23:38 [INFO] Skipping Phase 0 — reusing found: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000
|
||
2026-03-05 03:23:38 [INFO]
|
||
2026-03-05 03:23:38 [INFO] ------------------------------------------------------------------------
|
||
2026-03-05 03:23:38 [INFO] PHASE 1 — Internal Evaluation — 6 GPU Parallel
|
||
2026-03-05 03:23:38 [INFO] ------------------------------------------------------------------------
|
||
2026-03-05 03:23:38 [INFO] Submitted: GPU 5 — Calibration + Token NLL
|
||
2026-03-05 03:23:38 [INFO] Submitted: GPU 6 — Generation (15 prompts × 4 temps)
|
||
2026-03-05 03:23:38 [INFO] Submitted: GPU 7 — Repetition grid (12 × 5)
|
||
2026-03-05 03:23:38 [INFO] Submitted: GPU 2 — PPL: 3b_val.bin
|
||
2026-03-05 03:23:38 [INFO] Submitted: GPU 3 — PPL: korean_c4 + korean_val
|
||
2026-03-05 03:23:38 [INFO] Submitted: GPU 4 — PPL: hplt_ko + cc100_ko + PPL: 7 cosmo files + PPL: 7 remaining files
|
||
/usr/local/lib/python3.12/dist-packages/torch/library.py:356: UserWarning: Warning only once for all operators, other operators may also be overridden.
|
||
Overriding a previously registered kernel for the same operator and the same dispatch key
|
||
operator: flash_attn::_flash_attn_backward(Tensor dout, Tensor q, Tensor k, Tensor v, Tensor out, Tensor softmax_lse, Tensor(a6!)? dq, Tensor(a7!)? dk, Tensor(a8!)? dv, float dropout_p, float softmax_scale, bool causal, SymInt window_size_left, SymInt window_size_right, float softcap, Tensor? alibi_slopes, bool deterministic, Tensor? rng_state=None) -> Tensor
|
||
registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922
|
||
dispatch key: ADInplaceOrView
|
||
previous kernel: no debug info
|
||
new kernel: registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922 (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/core/dispatch/OperatorEntry.cpp:208.)
|
||
self.m.impl(
|
||
/usr/local/lib/python3.12/dist-packages/torch/library.py:356: UserWarning: Warning only once for all operators, other operators may also be overridden.
|
||
Overriding a previously registered kernel for the same operator and the same dispatch key
|
||
operator: flash_attn::_flash_attn_backward(Tensor dout, Tensor q, Tensor k, Tensor v, Tensor out, Tensor softmax_lse, Tensor(a6!)? dq, Tensor(a7!)? dk, Tensor(a8!)? dv, float dropout_p, float softmax_scale, bool causal, SymInt window_size_left, SymInt window_size_right, float softcap, Tensor? alibi_slopes, bool deterministic, Tensor? rng_state=None) -> Tensor
|
||
registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922
|
||
dispatch key: ADInplaceOrView
|
||
previous kernel: no debug info
|
||
new kernel: registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922 (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/core/dispatch/OperatorEntry.cpp:208.)
|
||
self.m.impl(
|
||
/usr/local/lib/python3.12/dist-packages/torch/library.py:356: UserWarning: Warning only once for all operators, other operators may also be overridden.
|
||
Overriding a previously registered kernel for the same operator and the same dispatch key
|
||
operator: flash_attn::_flash_attn_backward(Tensor dout, Tensor q, Tensor k, Tensor v, Tensor out, Tensor softmax_lse, Tensor(a6!)? dq, Tensor(a7!)? dk, Tensor(a8!)? dv, float dropout_p, float softmax_scale, bool causal, SymInt window_size_left, SymInt window_size_right, float softcap, Tensor? alibi_slopes, bool deterministic, Tensor? rng_state=None) -> Tensor
|
||
registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922
|
||
dispatch key: ADInplaceOrView
|
||
previous kernel: no debug info
|
||
new kernel: registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922 (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/core/dispatch/OperatorEntry.cpp:208.)
|
||
self.m.impl(
|
||
/usr/local/lib/python3.12/dist-packages/torch/library.py:356: UserWarning: Warning only once for all operators, other operators may also be overridden.
|
||
Overriding a previously registered kernel for the same operator and the same dispatch key
|
||
operator: flash_attn::_flash_attn_backward(Tensor dout, Tensor q, Tensor k, Tensor v, Tensor out, Tensor softmax_lse, Tensor(a6!)? dq, Tensor(a7!)? dk, Tensor(a8!)? dv, float dropout_p, float softmax_scale, bool causal, SymInt window_size_left, SymInt window_size_right, float softcap, Tensor? alibi_slopes, bool deterministic, Tensor? rng_state=None) -> Tensor
|
||
registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922
|
||
dispatch key: ADInplaceOrView
|
||
previous kernel: no debug info
|
||
new kernel: registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922 (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/core/dispatch/OperatorEntry.cpp:208.)
|
||
self.m.impl(
|
||
/usr/local/lib/python3.12/dist-packages/torch/library.py:356: UserWarning: Warning only once for all operators, other operators may also be overridden.
|
||
Overriding a previously registered kernel for the same operator and the same dispatch key
|
||
operator: flash_attn::_flash_attn_backward(Tensor dout, Tensor q, Tensor k, Tensor v, Tensor out, Tensor softmax_lse, Tensor(a6!)? dq, Tensor(a7!)? dk, Tensor(a8!)? dv, float dropout_p, float softmax_scale, bool causal, SymInt window_size_left, SymInt window_size_right, float softcap, Tensor? alibi_slopes, bool deterministic, Tensor? rng_state=None) -> Tensor
|
||
registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922
|
||
dispatch key: ADInplaceOrView
|
||
previous kernel: no debug info
|
||
new kernel: registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922 (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/core/dispatch/OperatorEntry.cpp:208.)
|
||
self.m.impl(
|
||
/usr/local/lib/python3.12/dist-packages/torch/library.py:356: UserWarning: Warning only once for all operators, other operators may also be overridden.
|
||
Overriding a previously registered kernel for the same operator and the same dispatch key
|
||
operator: flash_attn::_flash_attn_backward(Tensor dout, Tensor q, Tensor k, Tensor v, Tensor out, Tensor softmax_lse, Tensor(a6!)? dq, Tensor(a7!)? dk, Tensor(a8!)? dv, float dropout_p, float softmax_scale, bool causal, SymInt window_size_left, SymInt window_size_right, float softcap, Tensor? alibi_slopes, bool deterministic, Tensor? rng_state=None) -> Tensor
|
||
registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922
|
||
dispatch key: ADInplaceOrView
|
||
previous kernel: no debug info
|
||
new kernel: registered at /usr/local/lib/python3.12/dist-packages/torch/_library/custom_ops.py:922 (Triggered internally at /opt/pytorch/pytorch/aten/src/ATen/core/dispatch/OperatorEntry.cpp:208.)
|
||
self.m.impl(
|
||
[PPL_MULTI cuda:0] Loading model once for 2 files...
|
||
[PPL cuda:0] korean_c4: 15,159,838 tokens, 30.3 MB
|
||
[PPL cuda:0] Loading model for 3b...
|
||
[PPL cuda:0] 3b: 75,681,623 tokens, 151.4 MB
|
||
[CALIB cuda:0] Loading model...
|
||
[CALIB cuda:0] Using 50,000 tokens from 3b_val.bin
|
||
[PPL_MULTI cuda:0] Loading model once for 16 files...
|
||
[PPL cuda:0] hplt_ko: 16,165,735 tokens, 32.3 MB
|
||
[CALIB cuda:0] DONE top1=0.6875, top5=0.8164, top10=0.8593, entropy=1.5682, 3.8s
|
||
[NLL cuda:0] Loading model...
|
||
[NLL cuda:0] Using 50,000 tokens from 3b_val.bin
|
||
2026-03-05 03:24:05 [INFO] [DONE] GPU 5 — Calibration + Token NLL
|
||
2026-03-05 03:25:42 [ERROR] [FAILED] GPU 6 — Generation (15 prompts × 4 temps)
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
|
||
|
||
2026-03-05 03:25:42 [ERROR] [FAILED] GPU 7 — Repetition grid (12 × 5)
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
|
||
|
||
2026-03-05 03:25:42 [ERROR] [FAILED] GPU 2 — PPL: 3b_val.bin
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
|
||
|
||
2026-03-05 03:25:42 [ERROR] [FAILED] GPU 3 — PPL: korean_c4 + korean_val
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
|
||
|
||
2026-03-05 03:25:42 [ERROR] [FAILED] GPU 4 — PPL: hplt_ko + cc100_ko + PPL: 7 cosmo files + PPL: 7 remaining files
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 569, in run_phase1
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
|
||
|
||
2026-03-05 03:25:45 [INFO] Phase 1 complete: 1 succeeded, 5 failed
|
||
2026-03-05 03:25:45 [INFO] Phase 1 results saved: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0323/phase1_results.json
|
||
2026-03-05 03:25:45 [INFO] Phase 1 complete in 2m 6s.
|
||
2026-03-05 03:25:45 [INFO]
|
||
2026-03-05 03:25:45 [INFO] ------------------------------------------------------------------------
|
||
2026-03-05 03:25:45 [INFO] PHASE 2 — lm-eval Benchmarks — 6 GPU Parallel
|
||
2026-03-05 03:25:45 [INFO] ------------------------------------------------------------------------
|
||
2026-03-05 03:25:45 [INFO] Running 0-shot benchmarks on 8 GPUs ...
|
||
2026-03-05 03:25:45 [INFO] Submitted: [0-shot] GPU 2 — KoBEST: boolq + copa
|
||
2026-03-05 03:25:45 [INFO] Submitted: [0-shot] GPU 3 — KoBEST: hellaswag + sentineg
|
||
2026-03-05 03:25:45 [INFO] Submitted: [0-shot] GPU 4 — KoBEST: wic
|
||
2026-03-05 03:25:45 [INFO] Submitted: [0-shot] GPU 5 — HAE-RAE (all subtasks)
|
||
2026-03-05 03:25:45 [INFO] Submitted: [0-shot] GPU 6 — MMLU-KO part 1/2
|
||
2026-03-05 03:25:45 [INFO] Submitted: [0-shot] GPU 7 — MMLU-KO part 2/2
|
||
[Phase 2 [0-shot] GPU 2 — KoBEST: boolq + copa] Starting on cuda:2 — tasks: ['kobest_boolq', 'kobest_copa'], 0-shot
|
||
[Phase 2 [0-shot] GPU 3 — KoBEST: hellaswag + sentineg] Starting on cuda:3 — tasks: ['kobest_hellaswag', 'kobest_sentineg'], 0-shot
|
||
[Phase 2 [0-shot] GPU 4 — KoBEST: wic] Starting on cuda:4 — tasks: ['kobest_wic'], 0-shot
|
||
[Phase 2 [0-shot] GPU 5 — HAE-RAE (all subtasks)] Starting on cuda:5 — tasks: ['haerae'], 0-shot
|
||
[Phase 2 [0-shot] GPU 7 — MMLU-KO part 2/2] Starting on cuda:7 — tasks: ['global_mmlu_ko_high_school_statistics', 'global_mmlu_ko_high_school_us_history', 'global_mmlu_ko_high_school_world_history', 'global_mmlu_ko_human_aging', 'global_mmlu_ko_human_sexuality', 'global_mmlu_ko_international_law', 'global_mmlu_ko_jurisprudence', 'global_mmlu_ko_logical_fallacies', 'global_mmlu_ko_machine_learning', 'global_mmlu_ko_management', 'global_mmlu_ko_marketing', 'global_mmlu_ko_medical_genetics', 'global_mmlu_ko_miscellaneous', 'global_mmlu_ko_moral_disputes', 'global_mmlu_ko_moral_scenarios', 'global_mmlu_ko_nutrition', 'global_mmlu_ko_philosophy', 'global_mmlu_ko_prehistory', 'global_mmlu_ko_professional_accounting', 'global_mmlu_ko_professional_law', 'global_mmlu_ko_professional_medicine', 'global_mmlu_ko_professional_psychology', 'global_mmlu_ko_public_relations', 'global_mmlu_ko_security_studies', 'global_mmlu_ko_sociology', 'global_mmlu_ko_us_foreign_policy', 'global_mmlu_ko_virology', 'global_mmlu_ko_world_religions'], 0-shot
|
||
[Phase 2 [0-shot] GPU 6 — MMLU-KO part 1/2] Starting on cuda:6 — tasks: ['global_mmlu_ko_abstract_algebra', 'global_mmlu_ko_anatomy', 'global_mmlu_ko_astronomy', 'global_mmlu_ko_business_ethics', 'global_mmlu_ko_clinical_knowledge', 'global_mmlu_ko_college_biology', 'global_mmlu_ko_college_chemistry', 'global_mmlu_ko_college_computer_science', 'global_mmlu_ko_college_mathematics', 'global_mmlu_ko_college_medicine', 'global_mmlu_ko_college_physics', 'global_mmlu_ko_computer_security', 'global_mmlu_ko_conceptual_physics', 'global_mmlu_ko_econometrics', 'global_mmlu_ko_electrical_engineering', 'global_mmlu_ko_elementary_mathematics', 'global_mmlu_ko_formal_logic', 'global_mmlu_ko_global_facts', 'global_mmlu_ko_high_school_biology', 'global_mmlu_ko_high_school_chemistry', 'global_mmlu_ko_high_school_computer_science', 'global_mmlu_ko_high_school_european_history', 'global_mmlu_ko_high_school_geography', 'global_mmlu_ko_high_school_government_and_politics', 'global_mmlu_ko_high_school_macroeconomics', 'global_mmlu_ko_high_school_mathematics', 'global_mmlu_ko_high_school_microeconomics', 'global_mmlu_ko_high_school_physics', 'global_mmlu_ko_high_school_psychology'], 0-shot
|
||
2026-03-05 03:25:45 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:25:45 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:25:45 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:25:45 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:25:45 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:25:45 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:25:48 [INFO] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
|
||
2026-03-05 03:25:48 [INFO] Initializing hf model, with arguments: {'pretrained':
|
||
'/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000', 'dtype':
|
||
'bfloat16', 'device': 'cuda:0'}
|
||
2026-03-05 03:25:48 [INFO] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
|
||
2026-03-05 03:25:48 [INFO] Initializing hf model, with arguments: {'pretrained':
|
||
'/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000', 'dtype':
|
||
'bfloat16', 'device': 'cuda:0'}
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_statistics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_us_history' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_world_history' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_human_aging' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_human_sexuality' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_international_law' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_jurisprudence' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_logical_fallacies' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_machine_learning' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_management' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_marketing' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_medical_genetics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_miscellaneous' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_moral_disputes' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_moral_scenarios' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_nutrition' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_philosophy' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_prehistory' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_professional_accounting' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_professional_law' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_professional_medicine' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_professional_psychology' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_public_relations' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_security_studies' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_sociology' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_us_foreign_policy' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_virology' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_world_religions' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [INFO] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
|
||
[LM_EVAL] Starting on cuda:7 (CUDA_VISIBLE_DEVICES=7), tasks=['global_mmlu_ko_high_school_statistics', 'global_mmlu_ko_high_school_us_history', 'global_mmlu_ko_high_school_world_history', 'global_mmlu_ko_human_aging', 'global_mmlu_ko_human_sexuality', 'global_mmlu_ko_international_law', 'global_mmlu_ko_jurisprudence', 'global_mmlu_ko_logical_fallacies', 'global_mmlu_ko_machine_learning', 'global_mmlu_ko_management', 'global_mmlu_ko_marketing', 'global_mmlu_ko_medical_genetics', 'global_mmlu_ko_miscellaneous', 'global_mmlu_ko_moral_disputes', 'global_mmlu_ko_moral_scenarios', 'global_mmlu_ko_nutrition', 'global_mmlu_ko_philosophy', 'global_mmlu_ko_prehistory', 'global_mmlu_ko_professional_accounting', 'global_mmlu_ko_professional_law', 'global_mmlu_ko_professional_medicine', 'global_mmlu_ko_professional_psychology', 'global_mmlu_ko_public_relations', 'global_mmlu_ko_security_studies', 'global_mmlu_ko_sociology', 'global_mmlu_ko_us_foreign_policy', 'global_mmlu_ko_virology', 'global_mmlu_ko_world_religions'], num_fewshot=0
|
||
[LM_EVAL] No valid tasks to evaluate.
|
||
[Phase 2 [0-shot] GPU 7 — MMLU-KO part 2/2] Done.
|
||
2026-03-05 03:25:48 [INFO] Initializing hf model, with arguments: {'pretrained':
|
||
'/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000', 'dtype':
|
||
'bfloat16', 'device': 'cuda:0'}
|
||
2026-03-05 03:25:48 [INFO] [DONE] [0-shot] GPU 7 — MMLU-KO part 2/2
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_abstract_algebra' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_anatomy' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_astronomy' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_business_ethics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_clinical_knowledge' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_biology' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_chemistry' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_computer_science' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_mathematics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_medicine' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_physics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_computer_security' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_conceptual_physics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_econometrics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_electrical_engineering' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_elementary_mathematics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_formal_logic' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_global_facts' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_biology' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_chemistry' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_computer_science' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_european_history' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_geography' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_government_and_politics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_macroeconomics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_mathematics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_microeconomics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_physics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:25:48 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_psychology' not found in lm_eval registry — skipping.
|
||
[LM_EVAL] Starting on cuda:6 (CUDA_VISIBLE_DEVICES=6), tasks=['global_mmlu_ko_abstract_algebra', 'global_mmlu_ko_anatomy', 'global_mmlu_ko_astronomy', 'global_mmlu_ko_business_ethics', 'global_mmlu_ko_clinical_knowledge', 'global_mmlu_ko_college_biology', 'global_mmlu_ko_college_chemistry', 'global_mmlu_ko_college_computer_science', 'global_mmlu_ko_college_mathematics', 'global_mmlu_ko_college_medicine', 'global_mmlu_ko_college_physics', 'global_mmlu_ko_computer_security', 'global_mmlu_ko_conceptual_physics', 'global_mmlu_ko_econometrics', 'global_mmlu_ko_electrical_engineering', 'global_mmlu_ko_elementary_mathematics', 'global_mmlu_ko_formal_logic', 'global_mmlu_ko_global_facts', 'global_mmlu_ko_high_school_biology', 'global_mmlu_ko_high_school_chemistry', 'global_mmlu_ko_high_school_computer_science', 'global_mmlu_ko_high_school_european_history', 'global_mmlu_ko_high_school_geography', 'global_mmlu_ko_high_school_government_and_politics', 'global_mmlu_ko_high_school_macroeconomics', 'global_mmlu_ko_high_school_mathematics', 'global_mmlu_ko_high_school_microeconomics', 'global_mmlu_ko_high_school_physics', 'global_mmlu_ko_high_school_psychology'], num_fewshot=0
|
||
[LM_EVAL] No valid tasks to evaluate.
|
||
[Phase 2 [0-shot] GPU 6 — MMLU-KO part 1/2] Done.
|
||
2026-03-05 03:25:48 [INFO] [DONE] [0-shot] GPU 6 — MMLU-KO part 1/2
|
||
2026-03-05 03:25:48 [INFO] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
|
||
2026-03-05 03:25:48 [INFO] Initializing hf model, with arguments: {'pretrained':
|
||
'/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000', 'dtype':
|
||
'bfloat16', 'device': 'cuda:0'}
|
||
2026-03-05 03:25:53 [INFO] Using device 'cuda:0'
|
||
2026-03-05 03:25:53 [INFO] Using device 'cuda:0'
|
||
2026-03-05 03:25:53 [INFO] Using device 'cuda:0'
|
||
2026-03-05 03:25:53 [INFO] Using device 'cuda:0'
|
||
2026-03-05 03:25:53 [INFO] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}
|
||
2026-03-05 03:25:53 [INFO] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}
|
||
2026-03-05 03:25:53 [INFO] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}
|
||
2026-03-05 03:25:53 [INFO] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}
|
||
[LM_EVAL] Starting on cuda:2 (CUDA_VISIBLE_DEVICES=2), tasks=['kobest_boolq', 'kobest_copa'], num_fewshot=0
|
||
[LM_EVAL] Evaluating 2 task(s) together: ['kobest_boolq', 'kobest_copa']
|
||
|
||
Loading weights: 0%| | 0/171 [00:00<?, ?it/s]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 19599.55it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 9554.22it/s, Materializing param=lm_head.weight] [LM_EVAL] Starting on cuda:4 (CUDA_VISIBLE_DEVICES=4), tasks=['kobest_wic'], num_fewshot=0
|
||
[LM_EVAL] Evaluating 1 task(s) together: ['kobest_wic']
|
||
|
||
Loading weights: 0%| | 0/171 [00:00<?, ?it/s]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 13486.51it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 7570.95it/s, Materializing param=lm_head.weight] [LM_EVAL] Starting on cuda:5 (CUDA_VISIBLE_DEVICES=5), tasks=['haerae'], num_fewshot=0
|
||
[LM_EVAL] Evaluating 1 task(s) together: ['haerae']
|
||
|
||
Loading weights: 0%| | 0/171 [00:00<?, ?it/s]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 15196.75it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 7989.15it/s, Materializing param=lm_head.weight] [LM_EVAL] Starting on cuda:3 (CUDA_VISIBLE_DEVICES=3), tasks=['kobest_hellaswag', 'kobest_sentineg'], num_fewshot=0
|
||
[LM_EVAL] Evaluating 2 task(s) together: ['kobest_hellaswag', 'kobest_sentineg']
|
||
|
||
Loading weights: 0%| | 0/171 [00:00<?, ?it/s]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 15477.14it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 8224.13it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<01:08, 2.45it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<01:08, 2.45it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<01:08, 2.45it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<01:03, 2.66it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<01:03, 2.66it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<01:03, 2.66it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<01:06, 2.53it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<01:06, 2.53it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<01:06, 2.53it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<01:03, 2.66it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<01:03, 2.66it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<01:02, 2.66it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<01:02, 2.66it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<01:02, 2.66it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<01:02, 2.66it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<01:02, 2.66it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<01:02, 2.66it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<01:01, 2.66it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<01:01, 2.66it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<01:01, 2.66it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<01:01, 2.66it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<01:01, 2.66it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<01:01, 2.66it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<01:00, 2.66it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<01:00, 2.66it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<01:00, 2.66it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<01:00, 2.66it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:59, 2.66it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:59, 2.66it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:59, 2.66it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:59, 2.66it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:59, 2.66it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:59, 2.66it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<01:08, 2.45it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<01:08, 2.45it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<01:08, 2.45it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<01:08, 2.45it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<01:07, 2.45it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<01:07, 2.45it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<01:07, 2.45it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<01:07, 2.45it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<01:06, 2.45it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<01:06, 2.45it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<01:06, 2.45it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<01:06, 2.45it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<01:06, 2.45it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<01:06, 2.45it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<01:05, 2.45it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<01:05, 2.45it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<01:05, 2.45it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<01:05, 2.45it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<01:04, 2.45it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<01:04, 2.45it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<01:04, 2.45it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<01:04, 2.45it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:58, 2.66it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:58, 2.66it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:58, 2.66it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:58, 2.66it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<01:06, 2.53it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<01:06, 2.53it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<01:06, 2.53it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<01:06, 2.53it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<01:05, 2.53it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<01:05, 2.53it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<01:05, 2.53it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<01:05, 2.53it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<01:04, 2.53it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<01:04, 2.53it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<01:04, 2.53it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<01:04, 2.53it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<01:04, 2.53it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<01:04, 2.53it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<01:03, 2.53it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<01:03, 2.53it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<01:03, 2.53it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<01:03, 2.53it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:07, 19.65it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:07, 19.65it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:07, 19.65it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:07, 19.65it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:07, 19.65it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:07, 19.65it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:07, 19.65it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:45, 3.71it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:45, 3.71it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:45, 3.71it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<00:45, 3.71it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<00:45, 3.71it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<00:45, 3.71it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<00:45, 3.71it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<00:44, 3.71it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<00:44, 3.71it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<00:44, 3.71it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<00:44, 3.71it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<00:44, 3.71it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<00:44, 3.71it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<00:43, 3.71it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<00:43, 3.71it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<00:43, 3.71it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<00:43, 3.71it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<00:43, 3.71it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<00:43, 3.71it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:07, 19.65it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:07, 19.65it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<00:43, 3.71it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<00:43, 3.71it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:42, 3.71it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:42, 3.71it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:42, 3.71it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:42, 3.71it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:42, 3.71it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:42, 3.71it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:42, 3.71it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:42, 3.71it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:41, 3.71it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:41, 3.71it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:41, 3.71it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:41, 3.71it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:41, 3.71it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:41, 3.71it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:05, 26.04it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:01<00:07, 19.65it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:01<00:07, 19.65it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:09, 16.53it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:01<00:07, 19.65it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:01<00:07, 19.65it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:01<00:07, 19.65it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:01<00:07, 19.65it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:01<00:05, 26.04it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:01<00:05, 26.04it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:01<00:05, 26.04it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:01<00:05, 26.04it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:01<00:05, 26.04it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:01<00:05, 26.04it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:00<00:04, 35.47it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:01<00:05, 26.52it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:01<00:04, 32.25it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:01<00:04, 29.98it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:01<00:04, 31.69it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:00<00:03, 41.65it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:00<00:02, 45.18it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:00<00:02, 45.18it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:00<00:02, 45.18it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:00<00:02, 45.18it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:00<00:02, 45.18it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:00<00:02, 45.18it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:00<00:02, 45.18it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:00<00:02, 45.18it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:00<00:02, 45.18it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:00<00:02, 45.18it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:00<00:02, 45.18it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:01<00:03, 36.78it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:01<00:03, 37.39it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:01<00:03, 36.34it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:01<00:02, 45.18it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:01<00:02, 45.18it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:01<00:02, 45.18it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:01<00:02, 45.18it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:01<00:03, 39.43it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:01<00:03, 39.00it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:01<00:03, 39.32it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:01<00:03, 40.67it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:01<00:03, 40.67it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:01<00:03, 40.67it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:01<00:03, 40.67it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:01<00:03, 40.67it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:01<00:02, 45.62it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:01<00:03, 38.48it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:01<00:03, 39.68it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:01<00:02, 40.67it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:01<00:02, 40.67it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:01<00:02, 40.67it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:01<00:02, 40.67it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:01<00:02, 40.67it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:01<00:02, 40.67it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:01<00:02, 40.67it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:01<00:02, 40.67it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:01<00:03, 39.03it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:01<00:03, 39.03it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:01<00:03, 39.03it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:01<00:03, 39.03it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:01<00:03, 39.03it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:01<00:03, 39.03it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:01<00:03, 39.03it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:01<00:03, 39.03it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:01<00:03, 39.03it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:01<00:02, 39.03it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:01<00:02, 39.03it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:01<00:02, 42.51it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:01<00:02, 42.85it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:01<00:02, 39.03it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:01<00:02, 39.03it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:01<00:02, 39.03it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:01<00:02, 39.03it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:01<00:02, 39.03it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:01<00:02, 39.03it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:01<00:02, 39.03it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:01<00:02, 39.03it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:01<00:02, 42.10it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:01<00:02, 42.69it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:01<00:02, 48.56it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:01<00:02, 44.64it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:01<00:02, 40.58it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:01<00:02, 50.89it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:01<00:02, 50.89it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:01<00:02, 50.89it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:01<00:02, 45.75it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:01<00:02, 45.75it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:01<00:02, 45.75it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:02<00:02, 45.75it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:02<00:02, 45.75it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:01<00:02, 50.89it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:01<00:02, 50.89it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:01<00:02, 43.61it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:01<00:02, 43.61it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:01<00:02, 43.61it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:01<00:02, 46.68it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:01<00:02, 46.68it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:01<00:02, 46.68it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:01<00:02, 46.68it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:01<00:02, 46.68it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:01<00:02, 46.68it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:01<00:02, 46.68it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:01<00:02, 46.68it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:01<00:02, 46.68it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:01<00:02, 46.68it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:01<00:02, 46.68it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:01<00:02, 43.61it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:01<00:02, 43.61it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:02<00:02, 45.75it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:02<00:02, 45.75it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:02<00:02, 45.75it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:02<00:02, 45.75it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:02<00:02, 45.75it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:02<00:02, 45.75it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:02<00:02, 45.75it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:02<00:02, 45.75it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:01<00:02, 50.89it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:01<00:02, 50.89it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:01<00:02, 50.89it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:01<00:02, 50.89it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:01<00:02, 50.89it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:01<00:02, 50.89it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:01<00:01, 50.89it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:01<00:01, 50.89it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:02<00:02, 43.61it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:02<00:02, 43.61it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:02<00:02, 43.61it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:02<00:02, 43.61it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:02<00:02, 43.61it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:02<00:02, 43.61it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:02<00:02, 43.61it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:02<00:02, 43.61it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:02<00:02, 46.68it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:02<00:02, 46.68it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:01<00:02, 49.37it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:01<00:02, 49.37it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:01<00:02, 49.37it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:01<00:02, 49.37it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:01<00:02, 49.37it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:02<00:02, 44.69it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:02<00:02, 44.69it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:02<00:02, 44.69it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:02<00:02, 44.69it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:02<00:02, 44.69it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:02<00:02, 44.69it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:02<00:02, 44.69it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:02<00:02, 44.69it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:02<00:02, 44.69it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:02<00:02, 44.69it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:02<00:02, 44.69it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:02<00:02, 46.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:01<00:01, 49.37it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:01<00:01, 49.37it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:01<00:01, 49.37it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:01<00:01, 49.37it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:01<00:01, 49.37it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:01<00:01, 49.37it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:01<00:01, 49.37it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:01<00:01, 49.37it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:02<00:02, 45.90it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:02<00:02, 45.90it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:02<00:02, 45.90it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:02<00:02, 47.00it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:02<00:02, 45.90it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:02<00:02, 45.90it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:02<00:02, 44.89it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:02<00:02, 44.89it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:02<00:02, 44.89it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:02<00:02, 45.90it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:02<00:02, 45.90it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:02<00:01, 45.90it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:02<00:01, 45.90it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:02<00:01, 45.90it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:02<00:01, 45.90it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:02<00:01, 45.90it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:02<00:01, 45.90it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:01<00:01, 48.44it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:02<00:02, 44.89it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:02<00:02, 44.89it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:02<00:02, 44.89it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:02<00:02, 44.89it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:02<00:02, 44.89it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:02<00:02, 44.89it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:02<00:02, 44.89it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:02<00:02, 44.89it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:02<00:01, 44.89it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:02<00:01, 44.89it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:02<00:01, 48.73it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:02<00:02, 43.80it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:02<00:02, 43.80it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:02<00:02, 43.80it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:01<00:01, 45.16it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:01<00:01, 45.16it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:01<00:01, 45.16it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:01<00:01, 45.16it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:02<00:01, 45.16it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:02<00:01, 43.80it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:02<00:01, 43.80it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:02<00:01, 45.16it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:02<00:01, 45.16it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:02<00:01, 45.16it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:02<00:01, 45.16it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:02<00:01, 45.16it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:02<00:01, 45.16it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:02<00:01, 45.16it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:02<00:01, 45.16it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:02<00:01, 43.80it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:02<00:01, 43.80it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:02<00:01, 43.80it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:02<00:01, 43.80it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:02<00:01, 43.80it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:02<00:01, 43.80it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:02<00:01, 43.80it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:02<00:01, 43.80it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:02<00:01, 44.38it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:02<00:01, 44.20it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:02<00:01, 45.16it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:02<00:01, 45.16it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:02<00:01, 46.45it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:02<00:01, 48.52it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:02<00:01, 46.28it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:02<00:01, 45.97it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:02<00:01, 48.83it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:02<00:01, 51.25it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:02<00:01, 48.18it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:02<00:01, 47.56it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:02<00:01, 49.12it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:02<00:01, 50.40it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:02<00:01, 48.67it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:02<00:01, 48.02it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:02<00:01, 48.67it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:02<00:01, 48.67it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:02<00:01, 51.82it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:02<00:01, 51.82it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:02<00:01, 51.82it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:02<00:01, 51.82it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:02<00:01, 51.82it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:02<00:01, 51.82it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:02<00:01, 51.82it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:02<00:01, 48.07it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:02<00:01, 43.38it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:02<00:01, 48.67it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:02<00:01, 48.67it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:02<00:01, 48.67it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:02<00:01, 48.67it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:02<00:01, 48.67it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:02<00:01, 48.67it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:02<00:01, 48.67it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:02<00:01, 48.67it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:02<00:01, 51.82it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:02<00:01, 51.82it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:02<00:01, 47.93it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:03<00:01, 47.05it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:02<00:01, 51.82it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:02<00:01, 51.82it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:02<00:01, 46.38it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:03<00:01, 51.82it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:03<00:01, 51.82it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:03<00:01, 43.53it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:03<00:01, 43.53it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:03<00:01, 43.53it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:03<00:01, 43.53it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:03<00:01, 43.53it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:03<00:01, 43.53it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:03<00:01, 43.53it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:03<00:01, 47.57it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:03<00:01, 47.57it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:03<00:01, 43.53it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:03<00:01, 47.57it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:03<00:01, 43.53it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:03<00:01, 47.20it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:03<00:01, 47.20it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:03<00:01, 47.20it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:03<00:01, 47.57it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:03<00:01, 47.57it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:03<00:01, 43.53it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:03<00:01, 43.53it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:03<00:01, 47.57it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:03<00:01, 47.57it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:03<00:01, 47.20it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:03<00:01, 47.20it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:02<00:01, 46.57it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:03<00:01, 47.20it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:03<00:01, 47.20it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:03<00:01, 47.20it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:03<00:01, 47.20it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:03<00:01, 47.20it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:03<00:01, 47.20it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:03<00:00, 47.20it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:03<00:00, 47.20it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:03<00:01, 47.57it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:03<00:01, 47.57it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:03<00:01, 47.57it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:03<00:01, 47.57it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:03<00:00, 48.10it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:03<00:00, 48.10it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:03<00:00, 48.10it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:03<00:00, 48.87it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:03<00:00, 48.87it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:03<00:00, 48.87it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:03<00:00, 48.10it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:03<00:00, 48.10it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:03<00:01, 44.20it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:03<00:00, 48.87it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:03<00:00, 48.87it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:03<00:00, 48.10it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:03<00:00, 48.10it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:02<00:00, 46.58it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:03<00:01, 43.11it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:03<00:01, 43.11it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:03<00:01, 43.11it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:03<00:00, 43.11it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:03<00:00, 43.11it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:03<00:00, 43.11it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:03<00:00, 43.11it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:03<00:00, 43.11it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:03<00:00, 43.11it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:03<00:00, 48.10it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:03<00:00, 48.10it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:03<00:00, 48.10it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:03<00:00, 48.10it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:03<00:00, 45.01it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:03<00:00, 45.01it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:03<00:00, 45.01it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:03<00:00, 45.01it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:03<00:00, 45.01it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:03<00:00, 48.87it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:03<00:00, 48.87it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:03<00:00, 48.87it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:03<00:00, 48.87it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:03<00:00, 48.87it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:03<00:00, 48.87it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:02<00:00, 46.80it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:02<00:00, 46.80it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:02<00:00, 46.80it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:03<00:00, 45.01it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:03<00:00, 45.01it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:03<00:00, 43.11it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:03<00:00, 43.11it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:03<00:00, 45.01it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:03<00:00, 45.01it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:02<00:00, 46.80it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:02<00:00, 46.80it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:03<00:00, 43.11it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:03<00:00, 43.11it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:03<00:00, 47.53it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:03<00:00, 45.01it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:03<00:00, 45.01it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:03<00:00, 46.80it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:03<00:00, 46.80it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:03<00:00, 46.80it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:03<00:00, 46.80it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:03<00:00, 46.80it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:03<00:00, 46.80it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:03<00:00, 46.80it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:03<00:00, 46.80it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:03<00:00, 46.11it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:03<00:00, 46.11it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:03<00:00, 46.11it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:03<00:00, 46.11it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:03<00:00, 46.11it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:03<00:00, 46.11it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:03<00:00, 46.11it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:03<00:00, 46.11it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:03<00:00, 46.11it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:03<00:00, 46.11it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:03<00:00, 46.11it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:03<00:00, 45.73it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:03<00:00, 46.44it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:03<00:00, 41.59it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:03<00:00, 46.16it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:03<00:00, 46.96it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:03<00:00, 48.04it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:03<00:00, 46.63it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:03<00:00, 45.20it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:03<00:00, 47.87it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:03<00:00, 48.73it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:03<00:00, 48.80it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:03<00:00, 50.71it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:03<00:00, 49.49it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:03<00:00, 49.49it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:03<00:00, 49.49it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:03<00:00, 49.49it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:03<00:00, 49.49it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:03<00:00, 49.49it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:03<00:00, 49.49it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:03<00:00, 44.57it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:03<00:00, 44.58it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:03<00:00, 46.60it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:03<00:00, 49.49it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:03<00:00, 45.34it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:03<00:00, 45.34it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:03<00:00, 45.34it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:04<00:00, 45.33it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:04<00:00, 45.34it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:04<00:00, 45.34it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:04<00:00, 45.34it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:04<00:00, 45.34it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:04<00:00, 45.34it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:04<00:00, 45.34it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:04<00:00, 45.34it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:04<00:00, 45.34it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:04<00:00, 45.34it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:04<00:00, 45.34it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:03<00:00, 43.94it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:04<00:00, 49.49it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:04<00:00, 49.49it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:04<00:00, 49.49it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:04<00:00, 49.49it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:04<00:00, 49.49it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:04<00:00, 45.53it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:04<00:00, 45.53it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:04<00:00, 45.53it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:04<00:00, 45.53it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:04<00:00, 45.53it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:03<00:00, 45.86it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:04<00:00, 45.53it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:04<00:00, 45.53it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:03<00:00, 45.86it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:03<00:00, 45.86it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:04<00:00, 50.63it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:04<00:00, 45.53it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:04<00:00, 45.53it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:04<00:00, 51.70it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:03<00:00, 45.86it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:04<00:00, 45.53it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:04<00:00, 51.70it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:03<00:00, 45.86it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:04<00:00, 45.53it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:04<00:00, 51.70it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:03<00:00, 45.86it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:04<00:00, 51.70it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:03<00:00, 45.86it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:04<00:00, 51.70it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:04<00:00, 41.54it/s, Materializing param=model.norm.weight]
|
||
|
||
Loading weights: 99%|█████████▉| 170/171 [00:03<00:00, 45.86it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:03<00:00, 45.86it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:04<00:00, 41.56it/s, Materializing param=model.norm.weight]
|
||
|
||
Loading weights: 100%|██████████| 171/171 [00:03<00:00, 45.86it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:03<00:00, 45.86it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:04<00:00, 45.51it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:03<00:00, 45.41it/s, Materializing param=model.norm.weight]
|
||
|
||
Loading weights: 98%|█████████▊| 167/171 [00:04<00:00, 45.51it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:04<00:00, 45.51it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:04<00:00, 45.51it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:04<00:00, 45.51it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:04<00:00, 45.51it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:04<00:00, 45.51it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:04<00:00, 45.51it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:04<00:00, 45.51it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:04<00:00, 45.51it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:04<00:00, 45.51it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:04<00:00, 40.88it/s, Materializing param=model.norm.weight]
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
[1mLlamaForCausalLM LOAD REPORT[0m from: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000
|
||
Key | Status |
|
||
----------------------------------------------+---------+-
|
||
model.layers.{0...27}.self_attn.v_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.k_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.q_proj.weight | MISSING |
|
||
|
||
[3mNotes:
|
||
- MISSING[3m :those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.[0m
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
[1mLlamaForCausalLM LOAD REPORT[0m from: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000
|
||
Key | Status |
|
||
----------------------------------------------+---------+-
|
||
model.layers.{0...27}.self_attn.k_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.v_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.q_proj.weight | MISSING |
|
||
|
||
[3mNotes:
|
||
- MISSING[3m :those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.[0m
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
[1mLlamaForCausalLM LOAD REPORT[0m from: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000
|
||
Key | Status |
|
||
----------------------------------------------+---------+-
|
||
model.layers.{0...27}.self_attn.q_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.k_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.v_proj.weight | MISSING |
|
||
|
||
[3mNotes:
|
||
- MISSING[3m :those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.[0m
|
||
[1mLlamaForCausalLM LOAD REPORT[0m from: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000
|
||
Key | Status |
|
||
----------------------------------------------+---------+-
|
||
model.layers.{0...27}.self_attn.q_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.k_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.v_proj.weight | MISSING |
|
||
|
||
[3mNotes:
|
||
- MISSING[3m :those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.[0m
|
||
2026-03-05 03:26:01 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:01 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:02 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/b480e81024913f27783d2b05d2f0b10089db19ad/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:02 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/b480e81024913f27783d2b05d2f0b10089db19ad/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:02 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Downloading builder script: 0.00B [00:00, ?B/s]
|
||
Downloading builder script: 6.79kB [00:00, 31.2MB/s]
|
||
2026-03-05 03:26:02 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/README.md "HTTP/1.1 200 OK"
|
||
|
||
Downloading builder script: 0.00B [00:00, ?B/s]
|
||
Downloading builder script: 6.79kB [00:00, 35.8MB/s]
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:03 [WARNING] Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/HAERAE-HUB/HAE_RAE_BENCH/HAERAE-HUB/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/HAERAE-HUB/HAE_RAE_BENCH/revision/b480e81024913f27783d2b05d2f0b10089db19ad "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:03 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:03 [WARNING] Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/skt/kobest_v1/skt/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/skt/kobest_v1/skt/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/revision/a5ea15e3ac77ed694b79f6204eb31889a2ba989f "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/skt/kobest_v1/skt/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/revision/a5ea15e3ac77ed694b79f6204eb31889a2ba989f "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:04 [WARNING] Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/revision/a5ea15e3ac77ed694b79f6204eb31889a2ba989f "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=skt/kobest_v1 "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/revision/b480e81024913f27783d2b05d2f0b10089db19ad "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=skt/kobest_v1 "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:04 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=skt/kobest_v1 "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/wic?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=HAERAE-HUB/HAE_RAE_BENCH "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/wic/train.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:05 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq/train.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq%2Ftrain.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/HAERAE-HUB/HAE_RAE_BENCH/tree/b480e81024913f27783d2b05d2f0b10089db19ad/data?recursive=false&expand=false "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/wic%2Ftrain.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/hellaswag?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq%2Ftrain.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/tree/b480e81024913f27783d2b05d2f0b10089db19ad/data?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/wic%2Ftrain.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/hellaswag/train.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/hellaswag%2Ftrain.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq/test.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq%2Ftest.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq%2Ftest.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/wic/test.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:06 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/hellaswag%2Ftrain.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:07 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq/validation.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:07 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq%2Fvalidation.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:07 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/wic%2Ftest.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:07 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/hellaswag/test.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:07 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq%2Fvalidation.jsonl "HTTP/1.1 200 OK"
|
||
|
||
Generating train split: 0%| | 0/3665 [00:00<?, ? examples/s]
|
||
Generating train split: 100%|██████████| 3665/3665 [00:00<00:00, 785815.57 examples/s]
|
||
|
||
Generating test split: 0%| | 0/1404 [00:00<?, ? examples/s]2026-03-05 03:26:07 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/wic%2Ftest.jsonl "HTTP/1.1 200 OK"
|
||
|
||
Generating test split: 100%|██████████| 1404/1404 [00:00<00:00, 574237.23 examples/s]
|
||
|
||
Generating validation split: 0%| | 0/700 [00:00<?, ? examples/s]
|
||
Generating validation split: 100%|██████████| 700/700 [00:00<00:00, 386979.41 examples/s]
|
||
2026-03-05 03:26:07 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/hellaswag%2Ftest.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:07 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:07 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/wic/validation.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:07 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/hellaswag%2Ftest.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:08 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/data/standard_nomenclature-00000-of-00001-52814989a6843328.parquet "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:08 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/wic%2Fvalidation.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:08 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/hellaswag/validation.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:08 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/hellaswag%2Fvalidation.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:08 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/data/standard_nomenclature-00000-of-00001-52814989a6843328.parquet "HTTP/1.1 302 Found"
|
||
2026-03-05 03:26:08 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/hellaswag%2Fvalidation.jsonl "HTTP/1.1 200 OK"
|
||
|
||
Generating train split: 0%| | 0/2029 [00:00<?, ? examples/s]
|
||
Generating train split: 100%|██████████| 2029/2029 [00:00<00:00, 494379.16 examples/s]
|
||
|
||
Generating test split: 0%| | 0/500 [00:00<?, ? examples/s]
|
||
Generating test split: 100%|██████████| 500/500 [00:00<00:00, 252943.19 examples/s]
|
||
|
||
Generating validation split: 0%| | 0/500 [00:00<?, ? examples/s]
|
||
Generating validation split: 100%|██████████| 500/500 [00:00<00:00, 266068.51 examples/s]
|
||
|
||
Map: 0%| | 0/2029 [00:00<?, ? examples/s]2026-03-05 03:26:08 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/wic%2Fvalidation.jsonl "HTTP/1.1 200 OK"
|
||
|
||
Map: 100%|██████████| 2029/2029 [00:00<00:00, 30030.25 examples/s]
|
||
|
||
Generating train split: 0%| | 0/3318 [00:00<?, ? examples/s]
|
||
Generating train split: 100%|██████████| 3318/3318 [00:00<00:00, 948068.72 examples/s]
|
||
|
||
Generating test split: 0%| | 0/1260 [00:00<?, ? examples/s]
|
||
Generating test split: 100%|██████████| 1260/1260 [00:00<00:00, 619121.72 examples/s]
|
||
|
||
Generating validation split: 0%| | 0/610 [00:00<?, ? examples/s]
|
||
Generating validation split: 100%|██████████| 610/610 [00:00<00:00, 474856.24 examples/s]
|
||
|
||
Map: 0%| | 0/500 [00:00<?, ? examples/s]2026-03-05 03:26:08 [INFO] Selected tasks:
|
||
2026-03-05 03:26:08 [INFO] Task: kobest_wic (kobest/kobest_wic.yaml)
|
||
2026-03-05 03:26:08 [WARNING] Overwriting default num_fewshot of kobest_wic from None to 0
|
||
2026-03-05 03:26:08 [INFO] Building contexts for kobest_wic on rank 0...
|
||
|
||
Map: 100%|██████████| 500/500 [00:00<00:00, 28354.45 examples/s]
|
||
|
||
0%| | 0/1260 [00:00<?, ?it/s]
|
||
59%|█████▉ | 745/1260 [00:00<00:00, 7443.37it/s]2026-03-05 03:26:08 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/xet-read-token/b480e81024913f27783d2b05d2f0b10089db19ad "HTTP/1.1 200 OK"
|
||
|
||
100%|██████████| 1260/1260 [00:00<00:00, 3071.71it/s]
|
||
2026-03-05 03:26:08 [INFO] Running loglikelihood requests
|
||
|
||
Running loglikelihood requests: 0%| | 0/2520 [00:00<?, ?it/s]2026-03-05 03:26:09 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:09 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:09 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
|
||
Generating test split: 0 examples [00:00, ? examples/s]
|
||
Generating test split: 153 examples [00:00, 59941.02 examples/s]
|
||
|
||
Running loglikelihood requests: 0%| | 1/2520 [00:00<38:58, 1.08it/s]2026-03-05 03:26:10 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/skt/kobest_v1/skt/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 5%|▌ | 129/2520 [00:01<00:14, 164.05it/s]2026-03-05 03:26:10 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:10 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 10%|█ | 257/2520 [00:01<00:07, 322.34it/s]2026-03-05 03:26:10 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:10 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 15%|█▌ | 385/2520 [00:01<00:04, 474.81it/s]
|
||
Running loglikelihood requests: 20%|██ | 513/2520 [00:01<00:03, 615.93it/s]2026-03-05 03:26:10 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:10 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:10 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/b480e81024913f27783d2b05d2f0b10089db19ad/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:10 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=skt/kobest_v1 "HTTP/1.1 200 OK"
|
||
|
||
Running loglikelihood requests: 28%|██▊ | 705/2520 [00:01<00:02, 810.70it/s]
|
||
Running loglikelihood requests: 36%|███▌ | 897/2520 [00:01<00:01, 956.51it/s]2026-03-05 03:26:10 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:10 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 43%|████▎ | 1089/2520 [00:01<00:01, 1109.99it/s]
|
||
Running loglikelihood requests: 51%|█████ | 1281/2520 [00:01<00:01, 1183.96it/s]2026-03-05 03:26:11 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/copa?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:11 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:11 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/skt/kobest_v1/skt/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 58%|█████▊ | 1473/2520 [00:02<00:00, 1299.46it/s]
|
||
Running loglikelihood requests: 67%|██████▋ | 1700/2520 [00:02<00:00, 1532.40it/s]2026-03-05 03:26:11 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/copa/train.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 74%|███████▍ | 1868/2520 [00:02<00:00, 1503.97it/s]2026-03-05 03:26:11 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/copa%2Ftrain.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:11 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:11 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/copa%2Ftrain.jsonl "HTTP/1.1 200 OK"
|
||
|
||
Running loglikelihood requests: 81%|████████▏ | 2049/2520 [00:02<00:00, 1438.92it/s]
|
||
Running loglikelihood requests: 89%|████████▉ | 2241/2520 [00:02<00:00, 1502.63it/s]2026-03-05 03:26:11 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=skt/kobest_v1 "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:11 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/HAERAE-HUB/HAE_RAE_BENCH/HAERAE-HUB/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 95%|█████████▌| 2397/2520 [00:02<00:00, 1499.20it/s]
|
||
Running loglikelihood requests: 100%|██████████| 2520/2520 [00:02<00:00, 917.68it/s]
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/copa/test.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/copa%2Ftest.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/copa%2Ftest.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=HAERAE-HUB/HAE_RAE_BENCH "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg/train.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/copa/validation.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg%2Ftrain.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/copa%2Fvalidation.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/copa%2Fvalidation.jsonl "HTTP/1.1 200 OK"
|
||
|
||
Generating train split: 0%| | 0/3076 [00:00<?, ? examples/s]
|
||
Generating train split: 100%|██████████| 3076/3076 [00:00<00:00, 1217369.23 examples/s]
|
||
|
||
Generating test split: 0%| | 0/1000 [00:00<?, ? examples/s]
|
||
Generating test split: 100%|██████████| 1000/1000 [00:00<00:00, 564433.32 examples/s]
|
||
|
||
Generating validation split: 0%| | 0/500 [00:00<?, ? examples/s]
|
||
Generating validation split: 100%|██████████| 500/500 [00:00<00:00, 425127.10 examples/s]
|
||
2026-03-05 03:26:12 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg%2Ftrain.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:12 [INFO] Selected tasks:
|
||
2026-03-05 03:26:12 [INFO] Task: kobest_copa (kobest/kobest_copa.yaml)
|
||
2026-03-05 03:26:12 [INFO] Task: kobest_boolq (kobest/kobest_boolq.yaml)
|
||
2026-03-05 03:26:12 [WARNING] Overwriting default num_fewshot of kobest_copa from None to 0
|
||
2026-03-05 03:26:12 [WARNING] Overwriting default num_fewshot of kobest_boolq from None to 0
|
||
2026-03-05 03:26:12 [INFO] Building contexts for kobest_copa on rank 0...
|
||
|
||
0%| | 0/1000 [00:00<?, ?it/s]
|
||
100%|██████████| 1000/1000 [00:00<00:00, 203587.22it/s]
|
||
2026-03-05 03:26:12 [INFO] Building contexts for kobest_boolq on rank 0...
|
||
|
||
0%| | 0/1404 [00:00<?, ?it/s]2026-03-05 03:26:12 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:13 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg/test.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
12%|█▏ | 174/1404 [00:00<00:01, 661.56it/s]2026-03-05 03:26:13 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg%2Ftest.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:13 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:13 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg%2Ftest.jsonl "HTTP/1.1 200 OK"
|
||
|
||
34%|███▍ | 477/1404 [00:00<00:00, 1494.39it/s]Passed argument batch_size = auto:1. Detecting largest batch size
|
||
Determined largest batch size: 64
|
||
[LM_EVAL] Evaluation complete in 24.8s
|
||
[LM_EVAL] kobest_wic: acc=0.4881
|
||
[Phase 2 [0-shot] GPU 4 — KoBEST: wic] Done.
|
||
2026-03-05 03:26:13 [ERROR] [FAILED] [0-shot] GPU 4 — KoBEST: wic
|
||
concurrent.futures.process._RemoteTraceback:
|
||
"""
|
||
Traceback (most recent call last):
|
||
File "/usr/lib/python3.12/concurrent/futures/process.py", line 219, in _sendback_result
|
||
result_queue.put(_ResultItem(work_id, result=result,
|
||
File "/usr/lib/python3.12/multiprocessing/queues.py", line 393, in put
|
||
obj = _ForkingPickler.dumps(obj)
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/multiprocessing/reduction.py", line 51, in dumps
|
||
cls(buf, protocol).dump(obj)
|
||
_pickle.PicklingError: Can't pickle <function wic_doc_to_text at 0x79b7d79feca0>: import of module 'utils' failed
|
||
"""
|
||
|
||
The above exception was the direct cause of the following exception:
|
||
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 762, in run_phase2
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
_pickle.PicklingError: Can't pickle <function wic_doc_to_text at 0x79b7d79feca0>: import of module 'utils' failed
|
||
|
||
|
||
56%|█████▌ | 786/1404 [00:00<00:00, 2026.51it/s]
|
||
78%|███████▊ | 1095/1404 [00:00<00:00, 2367.95it/s]2026-03-05 03:26:13 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/data/rare_words-00000-of-00001-4f34314c09209524.parquet "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:13 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg/test_originated.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
100%|█████████▉| 1399/1404 [00:00<00:00, 2578.88it/s]
|
||
100%|██████████| 1404/1404 [00:00<00:00, 2109.35it/s]
|
||
2026-03-05 03:26:13 [INFO] Running loglikelihood requests
|
||
2026-03-05 03:26:13 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg%2Ftest_originated.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:13 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg%2Ftest_originated.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:13 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/data/rare_words-00000-of-00001-4f34314c09209524.parquet "HTTP/1.1 302 Found"
|
||
2026-03-05 03:26:13 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg/validation.jsonl "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:13 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg%2Fvalidation.jsonl "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:14 [INFO] HTTP Request: GET https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg%2Fvalidation.jsonl "HTTP/1.1 200 OK"
|
||
|
||
Generating train split: 0%| | 0/3649 [00:00<?, ? examples/s]
|
||
Generating train split: 100%|██████████| 3649/3649 [00:00<00:00, 1447145.92 examples/s]
|
||
|
||
Generating test split: 0%| | 0/397 [00:00<?, ? examples/s]
|
||
Generating test split: 100%|██████████| 397/397 [00:00<00:00, 508905.47 examples/s]
|
||
|
||
Generating test_originated split: 0%| | 0/397 [00:00<?, ? examples/s]
|
||
Generating test_originated split: 100%|██████████| 397/397 [00:00<00:00, 536449.32 examples/s]
|
||
|
||
Generating validation split: 0%| | 0/400 [00:00<?, ? examples/s]
|
||
Generating validation split: 100%|██████████| 400/400 [00:00<00:00, 559427.01 examples/s]
|
||
|
||
Running loglikelihood requests: 0%| | 0/4808 [00:00<?, ?it/s]2026-03-05 03:26:14 [INFO] Selected tasks:
|
||
2026-03-05 03:26:14 [INFO] Task: kobest_sentineg (kobest/kobest_sentineg.yaml)
|
||
2026-03-05 03:26:14 [INFO] Task: kobest_hellaswag (kobest/kobest_hellaswag.yaml)
|
||
2026-03-05 03:26:14 [WARNING] Overwriting default num_fewshot of kobest_sentineg from None to 0
|
||
2026-03-05 03:26:14 [WARNING] Overwriting default num_fewshot of kobest_hellaswag from None to 0
|
||
2026-03-05 03:26:14 [INFO] Building contexts for kobest_sentineg on rank 0...
|
||
|
||
0%| | 0/397 [00:00<?, ?it/s]
|
||
100%|██████████| 397/397 [00:00<00:00, 7428.42it/s]
|
||
2026-03-05 03:26:14 [INFO] Building contexts for kobest_hellaswag on rank 0...
|
||
|
||
Generating test split: 0 examples [00:00, ? examples/s]
|
||
Generating test split: 405 examples [00:00, 219497.75 examples/s]
|
||
|
||
0%| | 0/500 [00:00<?, ?it/s]
|
||
78%|███████▊ | 391/500 [00:00<00:00, 3906.17it/s]
|
||
100%|██████████| 500/500 [00:00<00:00, 3907.30it/s]
|
||
2026-03-05 03:26:14 [INFO] Running loglikelihood requests
|
||
2026-03-05 03:26:14 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 0%| | 0/2794 [00:00<?, ?it/s]2026-03-05 03:26:14 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:14 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/b480e81024913f27783d2b05d2f0b10089db19ad/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:15 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 0%| | 1/4808 [00:01<1:38:32, 1.23s/it]
|
||
Running loglikelihood requests: 1%|▏ | 65/4808 [00:01<01:11, 66.09it/s] 2026-03-05 03:26:15 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 3%|▎ | 129/4808 [00:01<00:34, 137.45it/s]
|
||
Running loglikelihood requests: 4%|▍ | 193/4808 [00:01<00:21, 210.39it/s]2026-03-05 03:26:15 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/HAERAE-HUB/HAE_RAE_BENCH/HAERAE-HUB/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 5%|▌ | 262/4808 [00:01<00:15, 294.89it/s]
|
||
Running loglikelihood requests: 0%| | 1/2794 [00:01<51:11, 1.10s/it]
|
||
Running loglikelihood requests: 7%|▋ | 351/4808 [00:01<00:10, 414.55it/s]
|
||
Running loglikelihood requests: 5%|▍ | 129/2794 [00:01<00:19, 136.25it/s]2026-03-05 03:26:16 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 9%|▉ | 449/4808 [00:01<00:09, 454.53it/s]
|
||
Running loglikelihood requests: 9%|▉ | 257/2794 [00:01<00:09, 265.80it/s]
|
||
Running loglikelihood requests: 12%|█▏ | 577/4808 [00:02<00:07, 537.32it/s]
|
||
Running loglikelihood requests: 14%|█▍ | 385/2794 [00:01<00:06, 391.50it/s]2026-03-05 03:26:16 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 15%|█▍ | 705/4808 [00:02<00:06, 596.24it/s]
|
||
Running loglikelihood requests: 18%|█▊ | 513/2794 [00:01<00:04, 511.63it/s]
|
||
Running loglikelihood requests: 23%|██▎ | 641/2794 [00:01<00:03, 619.36it/s]
|
||
Running loglikelihood requests: 17%|█▋ | 833/4808 [00:02<00:06, 640.35it/s]2026-03-05 03:26:16 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=HAERAE-HUB/HAE_RAE_BENCH "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 28%|██▊ | 769/2794 [00:01<00:02, 713.30it/s]
|
||
Running loglikelihood requests: 20%|█▉ | 961/4808 [00:02<00:05, 675.22it/s]
|
||
Running loglikelihood requests: 32%|███▏ | 897/2794 [00:02<00:02, 794.85it/s]2026-03-05 03:26:16 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 23%|██▎ | 1089/4808 [00:02<00:05, 702.26it/s]
|
||
Running loglikelihood requests: 37%|███▋ | 1025/2794 [00:02<00:02, 858.40it/s]
|
||
Running loglikelihood requests: 41%|████▏ | 1153/2794 [00:02<00:01, 903.46it/s]
|
||
Running loglikelihood requests: 25%|██▌ | 1217/4808 [00:02<00:04, 730.15it/s]2026-03-05 03:26:17 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 46%|████▌ | 1281/2794 [00:02<00:01, 957.49it/s]
|
||
Running loglikelihood requests: 28%|██▊ | 1345/4808 [00:03<00:04, 752.37it/s]
|
||
Running loglikelihood requests: 50%|█████ | 1409/2794 [00:02<00:01, 1007.13it/s]
|
||
Running loglikelihood requests: 31%|███ | 1473/4808 [00:03<00:04, 771.80it/s]
|
||
Running loglikelihood requests: 57%|█████▋ | 1594/2794 [00:02<00:00, 1222.76it/s]2026-03-05 03:26:17 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/data/loan_words-00000-of-00001-96ed08632b9a2ae8.parquet "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 62%|██████▏ | 1725/2794 [00:02<00:00, 1209.74it/s]
|
||
Running loglikelihood requests: 33%|███▎ | 1601/4808 [00:03<00:04, 789.02it/s]
|
||
Running loglikelihood requests: 66%|██████▋ | 1852/2794 [00:02<00:00, 1196.94it/s]
|
||
Running loglikelihood requests: 36%|███▌ | 1729/4808 [00:03<00:03, 807.96it/s]
|
||
Running loglikelihood requests: 71%|███████ | 1976/2794 [00:02<00:00, 1190.80it/s]2026-03-05 03:26:17 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/data/loan_words-00000-of-00001-96ed08632b9a2ae8.parquet "HTTP/1.1 302 Found"
|
||
|
||
Running loglikelihood requests: 79%|███████▉ | 2204/2794 [00:03<00:00, 1493.86it/s]
|
||
Running loglikelihood requests: 39%|███▊ | 1857/4808 [00:03<00:03, 822.85it/s]
|
||
Running loglikelihood requests: 89%|████████▉ | 2481/2794 [00:03<00:00, 1723.20it/s]
|
||
Running loglikelihood requests: 41%|████▏ | 1985/4808 [00:03<00:03, 837.73it/s]
|
||
Running loglikelihood requests: 100%|██████████| 2794/2794 [00:03<00:00, 844.03it/s]
|
||
|
||
Running loglikelihood requests: 44%|████▍ | 2113/4808 [00:04<00:03, 859.37it/s]
|
||
Running loglikelihood requests: 47%|████▋ | 2241/4808 [00:04<00:02, 883.17it/s]
|
||
Generating test split: 0 examples [00:00, ? examples/s]
|
||
Generating test split: 169 examples [00:00, 106145.16 examples/s]
|
||
|
||
Running loglikelihood requests: 49%|████▉ | 2369/4808 [00:04<00:02, 906.47it/s]
|
||
Running loglikelihood requests: 52%|█████▏ | 2497/4808 [00:04<00:02, 932.93it/s]2026-03-05 03:26:18 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 55%|█████▍ | 2625/4808 [00:04<00:02, 959.59it/s]
|
||
Running loglikelihood requests: 57%|█████▋ | 2753/4808 [00:04<00:02, 989.69it/s]2026-03-05 03:26:18 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 61%|██████▏ | 2945/4808 [00:04<00:01, 1115.93it/s]2026-03-05 03:26:18 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/b480e81024913f27783d2b05d2f0b10089db19ad/README.md "HTTP/1.1 200 OK"
|
||
|
||
Running loglikelihood requests: 65%|██████▌ | 3137/4808 [00:04<00:01, 1171.67it/s]
|
||
Running loglikelihood requests: 71%|███████ | 3393/4808 [00:05<00:00, 1492.62it/s]2026-03-05 03:26:19 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 307 Temporary Redirect"
|
||
Passed argument batch_size = auto:1. Detecting largest batch size
|
||
Determined largest batch size: 64
|
||
[LM_EVAL] Evaluation complete in 30.8s
|
||
[LM_EVAL] kobest_hellaswag: acc=0.1940, acc_norm=0.2160
|
||
[LM_EVAL] kobest_sentineg: acc=0.5013
|
||
[Phase 2 [0-shot] GPU 3 — KoBEST: hellaswag + sentineg] Done.
|
||
2026-03-05 03:26:19 [ERROR] [FAILED] [0-shot] GPU 3 — KoBEST: hellaswag + sentineg
|
||
concurrent.futures.process._RemoteTraceback:
|
||
"""
|
||
Traceback (most recent call last):
|
||
File "/usr/lib/python3.12/concurrent/futures/process.py", line 219, in _sendback_result
|
||
result_queue.put(_ResultItem(work_id, result=result,
|
||
File "/usr/lib/python3.12/multiprocessing/queues.py", line 393, in put
|
||
obj = _ForkingPickler.dumps(obj)
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/multiprocessing/reduction.py", line 51, in dumps
|
||
cls(buf, protocol).dump(obj)
|
||
_pickle.PicklingError: Can't pickle <function hellaswag_process_doc at 0x70db574db420>: import of module 'utils' failed
|
||
"""
|
||
|
||
The above exception was the direct cause of the following exception:
|
||
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 762, in run_phase2
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
_pickle.PicklingError: Can't pickle <function hellaswag_process_doc at 0x70db574db420>: import of module 'utils' failed
|
||
|
||
|
||
Running loglikelihood requests: 76%|███████▌ | 3650/4808 [00:05<00:00, 1756.64it/s]
|
||
Running loglikelihood requests: 81%|████████ | 3906/4808 [00:05<00:00, 1956.62it/s]
|
||
Running loglikelihood requests: 88%|████████▊ | 4226/4808 [00:05<00:00, 2181.32it/s]2026-03-05 03:26:19 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 94%|█████████▍| 4536/4808 [00:05<00:00, 2429.48it/s]2026-03-05 03:26:19 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/HAERAE-HUB/HAE_RAE_BENCH/HAERAE-HUB/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 100%|█████████▉| 4802/4808 [00:05<00:00, 2237.16it/s]
|
||
Running loglikelihood requests: 100%|██████████| 4808/4808 [00:05<00:00, 854.01it/s]
|
||
2026-03-05 03:26:20 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:20 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:20 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=HAERAE-HUB/HAE_RAE_BENCH "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:20 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:21 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
Passed argument batch_size = auto:1. Detecting largest batch size
|
||
Determined largest batch size: 64
|
||
[LM_EVAL] Evaluation complete in 32.8s
|
||
[LM_EVAL] kobest_boolq: acc=0.5014
|
||
[LM_EVAL] kobest_copa: acc=0.4940
|
||
[Phase 2 [0-shot] GPU 2 — KoBEST: boolq + copa] Done.
|
||
2026-03-05 03:26:21 [ERROR] [FAILED] [0-shot] GPU 2 — KoBEST: boolq + copa
|
||
concurrent.futures.process._RemoteTraceback:
|
||
"""
|
||
Traceback (most recent call last):
|
||
File "/usr/lib/python3.12/concurrent/futures/process.py", line 219, in _sendback_result
|
||
result_queue.put(_ResultItem(work_id, result=result,
|
||
File "/usr/lib/python3.12/multiprocessing/queues.py", line 393, in put
|
||
obj = _ForkingPickler.dumps(obj)
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/multiprocessing/reduction.py", line 51, in dumps
|
||
cls(buf, protocol).dump(obj)
|
||
_pickle.PicklingError: Can't pickle <function copa_doc_to_text at 0x72c23846c220>: import of module 'utils' failed
|
||
"""
|
||
|
||
The above exception was the direct cause of the following exception:
|
||
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 762, in run_phase2
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
_pickle.PicklingError: Can't pickle <function copa_doc_to_text at 0x72c23846c220>: import of module 'utils' failed
|
||
|
||
2026-03-05 03:26:21 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/data/history-00000-of-00001-9ad551d475d58413.parquet "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:21 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/data/history-00000-of-00001-9ad551d475d58413.parquet "HTTP/1.1 302 Found"
|
||
|
||
Generating test split: 0 examples [00:00, ? examples/s]
|
||
Generating test split: 188 examples [00:00, 112102.52 examples/s]
|
||
2026-03-05 03:26:22 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:22 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:22 [WARNING] Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:22 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/b480e81024913f27783d2b05d2f0b10089db19ad/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:23 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:23 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:23 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/HAERAE-HUB/HAE_RAE_BENCH/HAERAE-HUB/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:23 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:24 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:24 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=HAERAE-HUB/HAE_RAE_BENCH "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:24 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:24 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:25 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/data/general_knowledge-00000-of-00001-7418d1a0b8c4a615.parquet "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:25 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/data/general_knowledge-00000-of-00001-7418d1a0b8c4a615.parquet "HTTP/1.1 302 Found"
|
||
|
||
Generating test split: 0 examples [00:00, ? examples/s]
|
||
Generating test split: 176 examples [00:00, 106291.94 examples/s]
|
||
2026-03-05 03:26:25 [INFO] Selected tasks:
|
||
2026-03-05 03:26:25 [INFO] Group: haerae
|
||
2026-03-05 03:26:25 [INFO] ConfigurableGroup(group=haerae,group_alias=None): {'haerae_general_knowledge': ConfigurableTask(task_name=haerae_general_knowledge,output_type=multiple_choice,num_fewshot=None,num_samples=176), 'haerae_history': ConfigurableTask(task_name=haerae_history,output_type=multiple_choice,num_fewshot=None,num_samples=188), 'haerae_loan_word': ConfigurableTask(task_name=haerae_loan_word,output_type=multiple_choice,num_fewshot=None,num_samples=169), 'haerae_rare_word': ConfigurableTask(task_name=haerae_rare_word,output_type=multiple_choice,num_fewshot=None,num_samples=405), 'haerae_standard_nomenclature': ConfigurableTask(task_name=haerae_standard_nomenclature,output_type=multiple_choice,num_fewshot=None,num_samples=153)}
|
||
2026-03-05 03:26:25 [WARNING] Overwriting default num_fewshot of haerae_general_knowledge from None to 0
|
||
2026-03-05 03:26:25 [WARNING] Overwriting default num_fewshot of haerae_history from None to 0
|
||
2026-03-05 03:26:25 [WARNING] Overwriting default num_fewshot of haerae_loan_word from None to 0
|
||
2026-03-05 03:26:25 [WARNING] Overwriting default num_fewshot of haerae_rare_word from None to 0
|
||
2026-03-05 03:26:25 [WARNING] Overwriting default num_fewshot of haerae_standard_nomenclature from None to 0
|
||
2026-03-05 03:26:25 [INFO] Building contexts for haerae_general_knowledge on rank 0...
|
||
|
||
0%| | 0/176 [00:00<?, ?it/s]
|
||
100%|██████████| 176/176 [00:00<00:00, 3906.80it/s]
|
||
2026-03-05 03:26:26 [INFO] Building contexts for haerae_history on rank 0...
|
||
|
||
0%| | 0/188 [00:00<?, ?it/s]
|
||
100%|██████████| 188/188 [00:00<00:00, 3970.84it/s]
|
||
2026-03-05 03:26:26 [INFO] Building contexts for haerae_loan_word on rank 0...
|
||
|
||
0%| | 0/169 [00:00<?, ?it/s]
|
||
100%|██████████| 169/169 [00:00<00:00, 3977.52it/s]
|
||
2026-03-05 03:26:26 [INFO] Building contexts for haerae_rare_word on rank 0...
|
||
|
||
0%| | 0/405 [00:00<?, ?it/s]
|
||
99%|█████████▉| 401/405 [00:00<00:00, 4005.94it/s]
|
||
100%|██████████| 405/405 [00:00<00:00, 4001.61it/s]
|
||
2026-03-05 03:26:26 [INFO] Building contexts for haerae_standard_nomenclature on rank 0...
|
||
|
||
0%| | 0/153 [00:00<?, ?it/s]
|
||
100%|██████████| 153/153 [00:00<00:00, 3965.13it/s]
|
||
2026-03-05 03:26:26 [INFO] Running loglikelihood requests
|
||
|
||
Running loglikelihood requests: 0%| | 0/5455 [00:00<?, ?it/s]
|
||
Running loglikelihood requests: 0%| | 1/5455 [00:01<2:23:19, 1.58s/it]
|
||
Running loglikelihood requests: 1%| | 65/5455 [00:01<01:42, 52.72it/s] 2026-03-05 03:26:29 [ERROR] [FAILED] [0-shot] GPU 5 — HAE-RAE (all subtasks)
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 762, in run_phase2
|
||
result = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
|
||
|
||
2026-03-05 03:26:30 [INFO] Phase 2 (0-shot) complete: 2 succeeded, 4 failed
|
||
2026-03-05 03:26:30 [INFO] Attempting 5-shot benchmarks ...
|
||
2026-03-05 03:26:30 [INFO] Submitted: [5-shot] GPU 2 — KoBEST: boolq + copa
|
||
2026-03-05 03:26:30 [INFO] Submitted: [5-shot] GPU 3 — KoBEST: hellaswag + sentineg
|
||
2026-03-05 03:26:30 [INFO] Submitted: [5-shot] GPU 4 — KoBEST: wic
|
||
2026-03-05 03:26:30 [INFO] Submitted: [5-shot] GPU 5 — HAE-RAE (all subtasks)
|
||
2026-03-05 03:26:30 [INFO] Submitted: [5-shot] GPU 6 — MMLU-KO part 1/2
|
||
2026-03-05 03:26:30 [INFO] Submitted: [5-shot] GPU 7 — MMLU-KO part 2/2
|
||
[Phase 2 [5-shot] GPU 2 — KoBEST: boolq + copa] Starting on cuda:2 — tasks: ['kobest_boolq', 'kobest_copa'], 5-shot
|
||
[Phase 2 [5-shot] GPU 4 — KoBEST: wic] Starting on cuda:4 — tasks: ['kobest_wic'], 5-shot
|
||
[Phase 2 [5-shot] GPU 5 — HAE-RAE (all subtasks)] Starting on cuda:5 — tasks: ['haerae'], 5-shot
|
||
[Phase 2 [5-shot] GPU 3 — KoBEST: hellaswag + sentineg] Starting on cuda:3 — tasks: ['kobest_hellaswag', 'kobest_sentineg'], 5-shot
|
||
[Phase 2 [5-shot] GPU 6 — MMLU-KO part 1/2] Starting on cuda:6 — tasks: ['global_mmlu_ko_abstract_algebra', 'global_mmlu_ko_anatomy', 'global_mmlu_ko_astronomy', 'global_mmlu_ko_business_ethics', 'global_mmlu_ko_clinical_knowledge', 'global_mmlu_ko_college_biology', 'global_mmlu_ko_college_chemistry', 'global_mmlu_ko_college_computer_science', 'global_mmlu_ko_college_mathematics', 'global_mmlu_ko_college_medicine', 'global_mmlu_ko_college_physics', 'global_mmlu_ko_computer_security', 'global_mmlu_ko_conceptual_physics', 'global_mmlu_ko_econometrics', 'global_mmlu_ko_electrical_engineering', 'global_mmlu_ko_elementary_mathematics', 'global_mmlu_ko_formal_logic', 'global_mmlu_ko_global_facts', 'global_mmlu_ko_high_school_biology', 'global_mmlu_ko_high_school_chemistry', 'global_mmlu_ko_high_school_computer_science', 'global_mmlu_ko_high_school_european_history', 'global_mmlu_ko_high_school_geography', 'global_mmlu_ko_high_school_government_and_politics', 'global_mmlu_ko_high_school_macroeconomics', 'global_mmlu_ko_high_school_mathematics', 'global_mmlu_ko_high_school_microeconomics', 'global_mmlu_ko_high_school_physics', 'global_mmlu_ko_high_school_psychology'], 5-shot
|
||
[Phase 2 [5-shot] GPU 7 — MMLU-KO part 2/2] Starting on cuda:7 — tasks: ['global_mmlu_ko_high_school_statistics', 'global_mmlu_ko_high_school_us_history', 'global_mmlu_ko_high_school_world_history', 'global_mmlu_ko_human_aging', 'global_mmlu_ko_human_sexuality', 'global_mmlu_ko_international_law', 'global_mmlu_ko_jurisprudence', 'global_mmlu_ko_logical_fallacies', 'global_mmlu_ko_machine_learning', 'global_mmlu_ko_management', 'global_mmlu_ko_marketing', 'global_mmlu_ko_medical_genetics', 'global_mmlu_ko_miscellaneous', 'global_mmlu_ko_moral_disputes', 'global_mmlu_ko_moral_scenarios', 'global_mmlu_ko_nutrition', 'global_mmlu_ko_philosophy', 'global_mmlu_ko_prehistory', 'global_mmlu_ko_professional_accounting', 'global_mmlu_ko_professional_law', 'global_mmlu_ko_professional_medicine', 'global_mmlu_ko_professional_psychology', 'global_mmlu_ko_public_relations', 'global_mmlu_ko_security_studies', 'global_mmlu_ko_sociology', 'global_mmlu_ko_us_foreign_policy', 'global_mmlu_ko_virology', 'global_mmlu_ko_world_religions'], 5-shot
|
||
2026-03-05 03:26:31 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:26:31 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:26:31 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:26:31 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:26:31 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:26:31 [INFO] TensorFlow version 2.20.0 available.
|
||
2026-03-05 03:26:33 [INFO] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
|
||
2026-03-05 03:26:33 [INFO] Initializing hf model, with arguments: {'pretrained':
|
||
'/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000', 'dtype':
|
||
'bfloat16', 'device': 'cuda:0'}
|
||
2026-03-05 03:26:33 [INFO] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
|
||
2026-03-05 03:26:33 [INFO] Initializing hf model, with arguments: {'pretrained':
|
||
'/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000', 'dtype':
|
||
'bfloat16', 'device': 'cuda:0'}
|
||
2026-03-05 03:26:33 [INFO] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
|
||
2026-03-05 03:26:33 [INFO] Initializing hf model, with arguments: {'pretrained':
|
||
'/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000', 'dtype':
|
||
'bfloat16', 'device': 'cuda:0'}
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_statistics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_us_history' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_world_history' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_human_aging' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_human_sexuality' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_international_law' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_jurisprudence' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_logical_fallacies' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_machine_learning' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_management' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_marketing' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_medical_genetics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_miscellaneous' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_moral_disputes' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_moral_scenarios' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_nutrition' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_philosophy' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_prehistory' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_professional_accounting' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_professional_law' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_professional_medicine' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_professional_psychology' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_public_relations' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_security_studies' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_sociology' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_us_foreign_policy' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_virology' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_world_religions' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_abstract_algebra' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_anatomy' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_astronomy' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_business_ethics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_clinical_knowledge' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_biology' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_chemistry' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_computer_science' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_mathematics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_medicine' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_college_physics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_computer_security' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_conceptual_physics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_econometrics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_electrical_engineering' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_elementary_mathematics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_formal_logic' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_global_facts' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_biology' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_chemistry' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_computer_science' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_european_history' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_geography' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_government_and_politics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_macroeconomics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_mathematics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_microeconomics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_physics' not found in lm_eval registry — skipping.
|
||
2026-03-05 03:26:33 [WARNING] [LM_EVAL] Task 'global_mmlu_ko_high_school_psychology' not found in lm_eval registry — skipping.
|
||
[LM_EVAL] Starting on cuda:7 (CUDA_VISIBLE_DEVICES=7), tasks=['global_mmlu_ko_high_school_statistics', 'global_mmlu_ko_high_school_us_history', 'global_mmlu_ko_high_school_world_history', 'global_mmlu_ko_human_aging', 'global_mmlu_ko_human_sexuality', 'global_mmlu_ko_international_law', 'global_mmlu_ko_jurisprudence', 'global_mmlu_ko_logical_fallacies', 'global_mmlu_ko_machine_learning', 'global_mmlu_ko_management', 'global_mmlu_ko_marketing', 'global_mmlu_ko_medical_genetics', 'global_mmlu_ko_miscellaneous', 'global_mmlu_ko_moral_disputes', 'global_mmlu_ko_moral_scenarios', 'global_mmlu_ko_nutrition', 'global_mmlu_ko_philosophy', 'global_mmlu_ko_prehistory', 'global_mmlu_ko_professional_accounting', 'global_mmlu_ko_professional_law', 'global_mmlu_ko_professional_medicine', 'global_mmlu_ko_professional_psychology', 'global_mmlu_ko_public_relations', 'global_mmlu_ko_security_studies', 'global_mmlu_ko_sociology', 'global_mmlu_ko_us_foreign_policy', 'global_mmlu_ko_virology', 'global_mmlu_ko_world_religions'], num_fewshot=5
|
||
[LM_EVAL] No valid tasks to evaluate.
|
||
[Phase 2 [5-shot] GPU 7 — MMLU-KO part 2/2] Done.
|
||
2026-03-05 03:26:33 [INFO] [DONE] [5-shot] GPU 7 — MMLU-KO part 2/2
|
||
[LM_EVAL] Starting on cuda:6 (CUDA_VISIBLE_DEVICES=6), tasks=['global_mmlu_ko_abstract_algebra', 'global_mmlu_ko_anatomy', 'global_mmlu_ko_astronomy', 'global_mmlu_ko_business_ethics', 'global_mmlu_ko_clinical_knowledge', 'global_mmlu_ko_college_biology', 'global_mmlu_ko_college_chemistry', 'global_mmlu_ko_college_computer_science', 'global_mmlu_ko_college_mathematics', 'global_mmlu_ko_college_medicine', 'global_mmlu_ko_college_physics', 'global_mmlu_ko_computer_security', 'global_mmlu_ko_conceptual_physics', 'global_mmlu_ko_econometrics', 'global_mmlu_ko_electrical_engineering', 'global_mmlu_ko_elementary_mathematics', 'global_mmlu_ko_formal_logic', 'global_mmlu_ko_global_facts', 'global_mmlu_ko_high_school_biology', 'global_mmlu_ko_high_school_chemistry', 'global_mmlu_ko_high_school_computer_science', 'global_mmlu_ko_high_school_european_history', 'global_mmlu_ko_high_school_geography', 'global_mmlu_ko_high_school_government_and_politics', 'global_mmlu_ko_high_school_macroeconomics', 'global_mmlu_ko_high_school_mathematics', 'global_mmlu_ko_high_school_microeconomics', 'global_mmlu_ko_high_school_physics', 'global_mmlu_ko_high_school_psychology'], num_fewshot=5
|
||
[LM_EVAL] No valid tasks to evaluate.
|
||
[Phase 2 [5-shot] GPU 6 — MMLU-KO part 1/2] Done.
|
||
2026-03-05 03:26:33 [INFO] [DONE] [5-shot] GPU 6 — MMLU-KO part 1/2
|
||
2026-03-05 03:26:33 [INFO] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234 | Setting fewshot manual seed to 1234
|
||
2026-03-05 03:26:33 [INFO] Initializing hf model, with arguments: {'pretrained':
|
||
'/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000', 'dtype':
|
||
'bfloat16', 'device': 'cuda:0'}
|
||
2026-03-05 03:26:38 [INFO] Using device 'cuda:0'
|
||
2026-03-05 03:26:38 [INFO] Using device 'cuda:0'
|
||
2026-03-05 03:26:38 [INFO] Using device 'cuda:0'
|
||
2026-03-05 03:26:39 [INFO] Using device 'cuda:0'
|
||
2026-03-05 03:26:39 [INFO] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}
|
||
2026-03-05 03:26:39 [INFO] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}
|
||
2026-03-05 03:26:39 [INFO] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}
|
||
2026-03-05 03:26:39 [INFO] Model parallel was set to False, max memory was not set, and device map was set to {'': 'cuda:0'}
|
||
[LM_EVAL] Starting on cuda:2 (CUDA_VISIBLE_DEVICES=2), tasks=['kobest_boolq', 'kobest_copa'], num_fewshot=5
|
||
[LM_EVAL] Evaluating 2 task(s) together: ['kobest_boolq', 'kobest_copa']
|
||
|
||
Loading weights: 0%| | 0/171 [00:00<?, ?it/s]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 15650.39it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 7410.43it/s, Materializing param=lm_head.weight] [LM_EVAL] Starting on cuda:5 (CUDA_VISIBLE_DEVICES=5), tasks=['haerae'], num_fewshot=5
|
||
[LM_EVAL] Evaluating 1 task(s) together: ['haerae']
|
||
|
||
Loading weights: 0%| | 0/171 [00:00<?, ?it/s]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 12157.40it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 7073.03it/s, Materializing param=lm_head.weight] [LM_EVAL] Starting on cuda:3 (CUDA_VISIBLE_DEVICES=3), tasks=['kobest_hellaswag', 'kobest_sentineg'], num_fewshot=5
|
||
[LM_EVAL] Evaluating 2 task(s) together: ['kobest_hellaswag', 'kobest_sentineg']
|
||
|
||
Loading weights: 0%| | 0/171 [00:00<?, ?it/s]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 16980.99it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 8004.40it/s, Materializing param=lm_head.weight] [LM_EVAL] Starting on cuda:4 (CUDA_VISIBLE_DEVICES=4), tasks=['kobest_wic'], num_fewshot=5
|
||
[LM_EVAL] Evaluating 1 task(s) together: ['kobest_wic']
|
||
|
||
Loading weights: 0%| | 0/171 [00:00<?, ?it/s]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 13231.24it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 1/171 [00:00<00:00, 7476.48it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:22, 7.52it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:22, 7.52it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:22, 7.52it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<00:22, 7.52it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<00:22, 7.52it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<00:22, 7.52it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<00:22, 7.52it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<00:22, 7.52it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<00:22, 7.52it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:19, 8.79it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:19, 8.79it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:19, 8.79it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<00:19, 8.79it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<00:19, 8.79it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:29, 5.81it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:29, 5.81it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:29, 5.81it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<00:28, 5.81it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<00:27, 5.81it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<00:27, 5.81it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<00:27, 5.81it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<00:27, 5.81it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:21, 7.52it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:20, 7.52it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:20, 7.52it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:20, 7.52it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:20, 7.52it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:20, 7.52it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:20, 7.52it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:18, 8.79it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:17, 8.79it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:17, 8.79it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<00:27, 5.81it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<00:27, 5.81it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:17, 8.79it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:17, 8.79it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:17, 8.79it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:17, 8.79it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:17, 8.79it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:17, 8.79it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:27, 5.81it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:27, 5.81it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:25, 6.65it/s, Materializing param=lm_head.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:25, 6.65it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 1%| | 2/171 [00:00<00:25, 6.65it/s, Materializing param=model.embed_tokens.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:00<00:02, 61.82it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:00<00:02, 56.50it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<00:25, 6.65it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 3/171 [00:00<00:25, 6.65it/s, Materializing param=model.layers.0.input_layernorm.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<00:25, 6.65it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 2%|▏ | 4/171 [00:00<00:25, 6.65it/s, Materializing param=model.layers.0.mlp.down_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 3%|▎ | 5/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.0.mlp.gate_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▎ | 6/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.0.mlp.up_proj.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 4%|▍ | 7/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.0.post_attention_layernorm.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▍ | 8/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.0.self_attn.o_proj.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 5%|▌ | 9/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.1.input_layernorm.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▌ | 10/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.1.mlp.down_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 6%|▋ | 11/171 [00:00<00:24, 6.65it/s, Materializing param=model.layers.1.mlp.gate_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:23, 6.65it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 7%|▋ | 12/171 [00:00<00:23, 6.65it/s, Materializing param=model.layers.1.mlp.up_proj.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:23, 6.65it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 13/171 [00:00<00:23, 6.65it/s, Materializing param=model.layers.1.post_attention_layernorm.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:23, 6.65it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 8%|▊ | 14/171 [00:00<00:23, 6.65it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:04, 35.65it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.1.self_attn.o_proj.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 15/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.input_layernorm.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 9%|▉ | 16/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.mlp.down_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 10%|▉ | 17/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 11%|█ | 18/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 11%|█ | 19/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.post_attention_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 20/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.2.self_attn.o_proj.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 12%|█▏ | 21/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.3.input_layernorm.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 22/171 [00:00<00:03, 45.58it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:00<00:01, 80.02it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.3.mlp.down_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 13%|█▎ | 23/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.3.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:00<00:02, 53.55it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 14%|█▍ | 24/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.3.mlp.up_proj.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▍ | 25/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.3.post_attention_layernorm.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 15%|█▌ | 26/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.3.self_attn.o_proj.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▌ | 27/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.input_layernorm.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 16%|█▋ | 28/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.mlp.down_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:00<00:01, 74.19it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 17%|█▋ | 29/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.mlp.gate_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 18%|█▊ | 30/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.mlp.up_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 18%|█▊ | 31/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.post_attention_layernorm.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▊ | 32/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.4.self_attn.o_proj.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 19%|█▉ | 33/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.5.input_layernorm.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 20%|█▉ | 34/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.5.mlp.down_proj.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:00<00:01, 91.48it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:00<00:01, 88.85it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 20%|██ | 35/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.5.mlp.gate_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 21%|██ | 36/171 [00:00<00:02, 56.21it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.5.mlp.up_proj.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 37/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.5.post_attention_layernorm.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 22%|██▏ | 38/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.5.self_attn.o_proj.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 39/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.input_layernorm.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 23%|██▎ | 40/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.mlp.down_proj.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:00<00:01, 70.70it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 24%|██▍ | 41/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.mlp.gate_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 25%|██▍ | 42/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.mlp.up_proj.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 25%|██▌ | 43/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.post_attention_layernorm.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▌ | 44/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.6.self_attn.o_proj.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 26%|██▋ | 45/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.7.input_layernorm.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 46/171 [00:00<00:01, 78.75it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:00<00:01, 78.01it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.7.mlp.down_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 27%|██▋ | 47/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.7.mlp.gate_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 28%|██▊ | 48/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.7.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:00<00:01, 97.98it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:00<00:01, 89.54it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▊ | 49/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.7.post_attention_layernorm.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 29%|██▉ | 50/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.7.self_attn.o_proj.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|██▉ | 51/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.input_layernorm.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 30%|███ | 52/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.mlp.down_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 31%|███ | 53/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.mlp.gate_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 54/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 32%|███▏ | 55/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.post_attention_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 56/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.8.self_attn.o_proj.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 33%|███▎ | 57/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.9.input_layernorm.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 34%|███▍ | 58/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.9.mlp.down_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 35%|███▍ | 59/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.9.mlp.gate_proj.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:00<00:01, 99.55it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:00<00:01, 99.55it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:00<00:01, 99.55it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 35%|███▌ | 60/171 [00:00<00:01, 79.92it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:00<00:00, 101.42it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.9.mlp.up_proj.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▌ | 61/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.9.post_attention_layernorm.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 36%|███▋ | 62/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.9.self_attn.o_proj.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 63/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.input_layernorm.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 37%|███▋ | 64/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.mlp.down_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:00<00:01, 87.33it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:00<00:00, 101.82it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:01<00:01, 87.33it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:00<00:00, 99.55it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 38%|███▊ | 65/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.mlp.gate_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▊ | 66/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.mlp.up_proj.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 39%|███▉ | 67/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.post_attention_layernorm.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 40%|███▉ | 68/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.10.self_attn.o_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:01<00:01, 87.33it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 40%|████ | 69/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.11.input_layernorm.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 41%|████ | 70/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.11.mlp.down_proj.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:01<00:00, 101.82it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 71/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.11.mlp.gate_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 42%|████▏ | 72/171 [00:00<00:01, 90.30it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:01<00:01, 91.79it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:00<00:00, 104.44it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:00<00:01, 95.67it/s, Materializing param=model.layers.11.mlp.up_proj.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:00<00:01, 95.67it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 73/171 [00:00<00:01, 95.67it/s, Materializing param=model.layers.11.post_attention_layernorm.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:00<00:01, 95.67it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 43%|████▎ | 74/171 [00:00<00:01, 95.67it/s, Materializing param=model.layers.11.self_attn.o_proj.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:00<00:01, 95.67it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 75/171 [00:00<00:01, 95.67it/s, Materializing param=model.layers.12.input_layernorm.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:00<00:00, 95.67it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 44%|████▍ | 76/171 [00:00<00:00, 95.67it/s, Materializing param=model.layers.12.mlp.down_proj.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:01<00:00, 91.79it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:01<00:00, 91.79it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:01<00:00, 91.79it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:01<00:00, 91.79it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:01<00:00, 91.79it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:01<00:00, 91.79it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 45%|████▌ | 77/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.12.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:01<00:00, 104.44it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 46%|████▌ | 78/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.12.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 46%|████▌ | 79/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.12.post_attention_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 80/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.12.self_attn.o_proj.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 47%|████▋ | 81/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.13.input_layernorm.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 48%|████▊ | 82/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.13.mlp.down_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▊ | 83/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.13.mlp.gate_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 49%|████▉ | 84/171 [00:01<00:00, 95.67it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.13.mlp.up_proj.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|████▉ | 85/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.13.post_attention_layernorm.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 50%|█████ | 86/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.13.self_attn.o_proj.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████ | 87/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.input_layernorm.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 51%|█████▏ | 88/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:01<00:00, 109.22it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:01<00:00, 91.65it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 52%|█████▏ | 89/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.mlp.gate_proj.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 90/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.mlp.up_proj.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 53%|█████▎ | 91/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.post_attention_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 92/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.14.self_attn.o_proj.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 54%|█████▍ | 93/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.15.input_layernorm.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 55%|█████▍ | 94/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.15.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:01<00:00, 105.71it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:01<00:00, 101.44it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 95/171 [00:01<00:00, 99.94it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.15.mlp.gate_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 56%|█████▌ | 96/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.15.mlp.up_proj.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 97/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.15.post_attention_layernorm.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 57%|█████▋ | 98/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.15.self_attn.o_proj.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 99/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.input_layernorm.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 58%|█████▊ | 100/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.mlp.down_proj.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 59%|█████▉ | 101/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 60%|█████▉ | 102/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.mlp.up_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 60%|██████ | 103/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.post_attention_layernorm.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 61%|██████ | 104/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.16.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:01<00:00, 101.78it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 61%|██████▏ | 105/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.17.input_layernorm.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 62%|██████▏ | 106/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.17.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 107/171 [00:01<00:00, 99.53it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:01<00:00, 102.33it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.17.mlp.gate_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 63%|██████▎ | 108/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.17.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:01<00:00, 109.81it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:01<00:00, 107.42it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 64%|██████▎ | 109/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.17.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 64%|██████▍ | 110/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.17.self_attn.o_proj.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▍ | 111/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.input_layernorm.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 65%|██████▌ | 112/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:01<00:00, 97.87it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:01<00:00, 110.42it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 66%|██████▌ | 113/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.mlp.gate_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 114/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.mlp.up_proj.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 67%|██████▋ | 115/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.post_attention_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 116/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.18.self_attn.o_proj.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 68%|██████▊ | 117/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.19.input_layernorm.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 69%|██████▉ | 118/171 [00:01<00:00, 100.96it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:01<00:00, 113.42it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.19.mlp.down_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|██████▉ | 119/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.19.mlp.gate_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 70%|███████ | 120/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.19.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 71%|███████ | 121/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.19.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 71%|███████▏ | 122/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.19.self_attn.o_proj.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 72%|███████▏ | 123/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 73%|███████▎ | 124/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.mlp.down_proj.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:01<00:00, 100.34it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 73%|███████▎ | 125/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.mlp.gate_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▎ | 126/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.mlp.up_proj.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 74%|███████▍ | 127/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.post_attention_layernorm.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▍ | 128/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.20.self_attn.o_proj.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 75%|███████▌ | 129/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.21.input_layernorm.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 76%|███████▌ | 130/171 [00:01<00:00, 85.21it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:01<00:00, 106.87it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.21.mlp.down_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 131/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.21.mlp.gate_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 77%|███████▋ | 132/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.21.mlp.up_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:01<00:00, 111.95it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 78%|███████▊ | 133/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.21.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 78%|███████▊ | 134/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.21.self_attn.o_proj.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 79%|███████▉ | 135/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 80%|███████▉ | 136/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.mlp.down_proj.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 80%|████████ | 137/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 81%|████████ | 138/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:01<00:00, 111.93it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 111.93it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 111.93it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 100.73it/s, Materializing param=model.norm.weight]
|
||
|
||
Loading weights: 91%|█████████ | 156/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 81%|████████▏ | 139/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.post_attention_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 140/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.22.self_attn.o_proj.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 82%|████████▏ | 141/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.23.input_layernorm.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 83%|████████▎ | 142/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.23.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▎ | 143/171 [00:01<00:00, 91.21it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:01<00:00, 100.93it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.23.mlp.gate_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 84%|████████▍ | 144/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.23.mlp.up_proj.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▍ | 145/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.23.post_attention_layernorm.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 85%|████████▌ | 146/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.23.self_attn.o_proj.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 86%|████████▌ | 147/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.input_layernorm.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 87%|████████▋ | 148/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.mlp.down_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:01<00:00, 107.44it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:01<00:00, 128.52it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:01<00:00, 128.52it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:01<00:00, 128.52it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:01<00:00, 128.52it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:01<00:00, 128.52it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 128.52it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 128.52it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 92.91it/s, Materializing param=model.norm.weight]
|
||
|
||
Loading weights: 99%|█████████▉| 169/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:01<00:00, 97.50it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 97.50it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 97.50it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 94.18it/s, Materializing param=model.norm.weight]
|
||
|
||
Loading weights: 87%|████████▋ | 149/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 87%|████████▋ | 149/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.mlp.gate_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 150/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.mlp.up_proj.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 88%|████████▊ | 151/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.post_attention_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 152/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.24.self_attn.o_proj.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 89%|████████▉ | 153/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.input_layernorm.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 90%|█████████ | 154/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.mlp.down_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 155/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.mlp.gate_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 91%|█████████ | 156/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.mlp.up_proj.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 157/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.post_attention_layernorm.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 92%|█████████▏| 158/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.25.self_attn.o_proj.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 93%|█████████▎| 159/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.input_layernorm.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▎| 160/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.mlp.down_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 94%|█████████▍| 161/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.mlp.gate_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▍| 162/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.mlp.up_proj.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 95%|█████████▌| 163/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.post_attention_layernorm.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▌| 164/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.26.self_attn.o_proj.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 96%|█████████▋| 165/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.input_layernorm.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 97%|█████████▋| 166/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.mlp.down_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 167/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.mlp.gate_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 98%|█████████▊| 168/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.mlp.up_proj.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 169/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.post_attention_layernorm.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 99%|█████████▉| 170/171 [00:01<00:00, 100.15it/s, Materializing param=model.layers.27.self_attn.o_proj.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 100.15it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 100.15it/s, Materializing param=model.norm.weight]
|
||
Loading weights: 100%|██████████| 171/171 [00:01<00:00, 94.72it/s, Materializing param=model.norm.weight]
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
[1mLlamaForCausalLM LOAD REPORT[0m from: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000
|
||
Key | Status |
|
||
----------------------------------------------+---------+-
|
||
model.layers.{0...27}.self_attn.v_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.k_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.q_proj.weight | MISSING |
|
||
|
||
[3mNotes:
|
||
- MISSING[3m :those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.[0m
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
[1mLlamaForCausalLM LOAD REPORT[0m from: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000
|
||
Key | Status |
|
||
----------------------------------------------+---------+-
|
||
model.layers.{0...27}.self_attn.q_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.v_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.k_proj.weight | MISSING |
|
||
|
||
[3mNotes:
|
||
- MISSING[3m :those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.[0m
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
[1mLlamaForCausalLM LOAD REPORT[0m from: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000
|
||
Key | Status |
|
||
----------------------------------------------+---------+-
|
||
model.layers.{0...27}.self_attn.v_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.k_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.q_proj.weight | MISSING |
|
||
|
||
[3mNotes:
|
||
- MISSING[3m :those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.[0m
|
||
The tied weights mapping and config for this model specifies to tie model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
|
||
[1mLlamaForCausalLM LOAD REPORT[0m from: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0318/hf_3b_checkpoint-0057000
|
||
Key | Status |
|
||
----------------------------------------------+---------+-
|
||
model.layers.{0...27}.self_attn.k_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.q_proj.weight | MISSING |
|
||
model.layers.{0...27}.self_attn.v_proj.weight | MISSING |
|
||
|
||
[3mNotes:
|
||
- MISSING[3m :those params were newly initialized because missing from the checkpoint. Consider training on your downstream task.[0m
|
||
2026-03-05 03:26:48 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:48 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:48 [WARNING] Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:48 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/b480e81024913f27783d2b05d2f0b10089db19ad/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:49 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:49 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:49 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:49 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:49 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:49 [WARNING] Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
||
2026-03-05 03:26:49 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:49 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:49 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:49 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:49 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/HAERAE-HUB/HAE_RAE_BENCH/HAERAE-HUB/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:49 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/HAERAE-HUB/HAE_RAE_BENCH/revision/b480e81024913f27783d2b05d2f0b10089db19ad "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/revision/b480e81024913f27783d2b05d2f0b10089db19ad "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/skt/kobest_v1/skt/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/skt/kobest_v1/skt/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/revision/a5ea15e3ac77ed694b79f6204eb31889a2ba989f "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/skt/kobest_v1/skt/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/revision/a5ea15e3ac77ed694b79f6204eb31889a2ba989f "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/revision/a5ea15e3ac77ed694b79f6204eb31889a2ba989f "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=HAERAE-HUB/HAE_RAE_BENCH "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:50 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=skt/kobest_v1 "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/HAERAE-HUB/HAE_RAE_BENCH/tree/b480e81024913f27783d2b05d2f0b10089db19ad/data?recursive=false&expand=false "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=skt/kobest_v1 "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/tree/b480e81024913f27783d2b05d2f0b10089db19ad/data?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=skt/kobest_v1 "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/hellaswag?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/boolq?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:51 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:52 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/wic?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:52 [INFO] Selected tasks:
|
||
2026-03-05 03:26:52 [INFO] Task: kobest_wic (kobest/kobest_wic.yaml)
|
||
2026-03-05 03:26:52 [WARNING] Overwriting default num_fewshot of kobest_wic from None to 5
|
||
2026-03-05 03:26:52 [INFO] Building contexts for kobest_wic on rank 0...
|
||
|
||
0%| | 0/1260 [00:00<?, ?it/s]2026-03-05 03:26:52 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:52 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/b480e81024913f27783d2b05d2f0b10089db19ad/README.md "HTTP/1.1 200 OK"
|
||
|
||
10%|█ | 128/1260 [00:00<00:00, 1274.66it/s]
|
||
20%|██ | 258/1260 [00:00<00:00, 1288.29it/s]2026-03-05 03:26:52 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
31%|███ | 389/1260 [00:00<00:00, 1297.11it/s]
|
||
41%|████▏ | 520/1260 [00:00<00:00, 1300.31it/s]2026-03-05 03:26:52 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
52%|█████▏ | 651/1260 [00:00<00:00, 1300.64it/s]
|
||
62%|██████▏ | 783/1260 [00:00<00:00, 1305.30it/s]2026-03-05 03:26:52 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/HAERAE-HUB/HAE_RAE_BENCH/HAERAE-HUB/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:52 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:52 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:52 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:52 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:52 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/skt/kobest_v1/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/README.md "HTTP/1.1 200 OK"
|
||
|
||
73%|███████▎ | 914/1260 [00:00<00:00, 757.35it/s] 2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
|
||
83%|████████▎ | 1041/1260 [00:01<00:00, 865.76it/s]2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
|
||
93%|█████████▎| 1170/1260 [00:01<00:00, 963.53it/s]2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/skt/kobest_v1/skt/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
|
||
100%|██████████| 1260/1260 [00:01<00:00, 1054.36it/s]
|
||
2026-03-05 03:26:53 [INFO] Running loglikelihood requests
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=HAERAE-HUB/HAE_RAE_BENCH "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/skt/kobest_v1/skt/kobest_v1.py "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=skt/kobest_v1 "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=skt/kobest_v1 "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:53 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/skt/kobest_v1/resolve/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:54 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/sentineg?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:54 [INFO] Selected tasks:
|
||
2026-03-05 03:26:54 [INFO] Task: kobest_sentineg (kobest/kobest_sentineg.yaml)
|
||
2026-03-05 03:26:54 [INFO] Task: kobest_hellaswag (kobest/kobest_hellaswag.yaml)
|
||
2026-03-05 03:26:54 [WARNING] Overwriting default num_fewshot of kobest_sentineg from None to 5
|
||
2026-03-05 03:26:54 [WARNING] Overwriting default num_fewshot of kobest_hellaswag from None to 5
|
||
2026-03-05 03:26:54 [INFO] Building contexts for kobest_sentineg on rank 0...
|
||
|
||
0%| | 0/397 [00:00<?, ?it/s]2026-03-05 03:26:54 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:54 [INFO] HTTP Request: GET https://huggingface.co/api/datasets/skt/kobest_v1/tree/a5ea15e3ac77ed694b79f6204eb31889a2ba989f/copa?recursive=false&expand=false "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:54 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/b480e81024913f27783d2b05d2f0b10089db19ad/README.md "HTTP/1.1 200 OK"
|
||
2026-03-05 03:26:54 [INFO] Selected tasks:
|
||
2026-03-05 03:26:54 [INFO] Task: kobest_copa (kobest/kobest_copa.yaml)
|
||
2026-03-05 03:26:54 [INFO] Task: kobest_boolq (kobest/kobest_boolq.yaml)
|
||
2026-03-05 03:26:54 [WARNING] Overwriting default num_fewshot of kobest_copa from None to 5
|
||
2026-03-05 03:26:54 [WARNING] Overwriting default num_fewshot of kobest_boolq from None to 5
|
||
2026-03-05 03:26:54 [INFO] Building contexts for kobest_copa on rank 0...
|
||
|
||
32%|███▏ | 129/397 [00:00<00:00, 1280.88it/s]
|
||
0%| | 0/1000 [00:00<?, ?it/s]
|
||
100%|██████████| 1000/1000 [00:00<00:00, 66183.36it/s]
|
||
2026-03-05 03:26:54 [INFO] Building contexts for kobest_boolq on rank 0...
|
||
|
||
Running loglikelihood requests: 0%| | 0/2520 [00:00<?, ?it/s]2026-03-05 03:26:54 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
0%| | 0/1404 [00:00<?, ?it/s]
|
||
65%|██████▍ | 258/397 [00:00<00:00, 544.65it/s] 2026-03-05 03:26:54 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
4%|▎ | 51/1404 [00:00<00:02, 506.41it/s]
|
||
98%|█████████▊| 388/397 [00:00<00:00, 741.00it/s]
|
||
100%|██████████| 397/397 [00:00<00:00, 734.18it/s]
|
||
2026-03-05 03:26:54 [INFO] Building contexts for kobest_hellaswag on rank 0...
|
||
|
||
0%| | 0/500 [00:00<?, ?it/s]
|
||
7%|▋ | 103/1404 [00:00<00:02, 510.32it/s]2026-03-05 03:26:54 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/HAERAE-HUB/HAE_RAE_BENCH/HAERAE-HUB/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
13%|█▎ | 67/500 [00:00<00:00, 664.02it/s]
|
||
11%|█ | 155/1404 [00:00<00:02, 510.56it/s]
|
||
27%|██▋ | 135/500 [00:00<00:00, 670.89it/s]
|
||
15%|█▍ | 207/1404 [00:00<00:02, 510.61it/s]2026-03-05 03:26:54 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
41%|████ | 203/500 [00:00<00:00, 673.71it/s]
|
||
19%|█▊ | 260/1404 [00:00<00:02, 516.13it/s]
|
||
54%|█████▍ | 271/500 [00:00<00:00, 675.03it/s]
|
||
22%|██▏ | 313/1404 [00:00<00:02, 520.41it/s]2026-03-05 03:26:55 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
|
||
68%|██████▊ | 339/500 [00:00<00:00, 675.32it/s]
|
||
26%|██▌ | 367/1404 [00:00<00:01, 524.05it/s]
|
||
81%|████████▏ | 407/500 [00:00<00:00, 675.10it/s]
|
||
30%|██▉ | 421/1404 [00:00<00:01, 527.06it/s]2026-03-05 03:26:55 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=HAERAE-HUB/HAE_RAE_BENCH "HTTP/1.1 404 Not Found"
|
||
|
||
95%|█████████▌| 475/500 [00:00<00:00, 675.62it/s]
|
||
34%|███▍ | 475/1404 [00:00<00:01, 529.57it/s]
|
||
100%|██████████| 500/500 [00:00<00:00, 674.62it/s]
|
||
2026-03-05 03:26:55 [INFO] Running loglikelihood requests
|
||
|
||
38%|███▊ | 529/1404 [00:01<00:01, 531.06it/s]2026-03-05 03:26:55 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
42%|████▏ | 583/1404 [00:01<00:01, 532.16it/s]
|
||
45%|████▌ | 637/1404 [00:01<00:01, 532.66it/s]2026-03-05 03:26:55 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
|
||
49%|████▉ | 691/1404 [00:01<00:01, 533.55it/s]
|
||
53%|█████▎ | 745/1404 [00:01<00:01, 533.95it/s]2026-03-05 03:26:55 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
57%|█████▋ | 799/1404 [00:01<00:01, 531.79it/s]
|
||
Running loglikelihood requests: 0%| | 1/2520 [00:01<1:13:18, 1.75s/it]
|
||
61%|██████ | 853/1404 [00:01<00:01, 528.11it/s]2026-03-05 03:26:56 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:56 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/b480e81024913f27783d2b05d2f0b10089db19ad/README.md "HTTP/1.1 200 OK"
|
||
|
||
65%|██████▍ | 907/1404 [00:01<00:00, 529.75it/s]
|
||
Running loglikelihood requests: 3%|▎ | 65/2520 [00:01<00:54, 44.90it/s]
|
||
68%|██████▊ | 961/1404 [00:01<00:00, 531.72it/s]2026-03-05 03:26:56 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
72%|███████▏ | 1015/1404 [00:01<00:00, 532.96it/s]
|
||
76%|███████▌ | 1069/1404 [00:02<00:00, 534.02it/s]
|
||
Running loglikelihood requests: 5%|▌ | 129/2520 [00:02<00:26, 89.50it/s]2026-03-05 03:26:56 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
80%|███████▉ | 1123/1404 [00:02<00:00, 523.15it/s]
|
||
84%|████████▍ | 1176/1404 [00:02<00:00, 518.87it/s]
|
||
Running loglikelihood requests: 8%|▊ | 193/2520 [00:02<00:17, 131.98it/s]2026-03-05 03:26:56 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/HAERAE-HUB/HAE_RAE_BENCH/HAERAE-HUB/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 0%| | 0/2794 [00:00<?, ?it/s]
|
||
88%|████████▊ | 1229/1404 [00:02<00:00, 521.06it/s]
|
||
91%|█████████▏| 1283/1404 [00:02<00:00, 525.22it/s]
|
||
Running loglikelihood requests: 10%|█ | 257/2520 [00:02<00:13, 169.04it/s]2026-03-05 03:26:56 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
95%|█████████▌| 1337/1404 [00:02<00:00, 528.57it/s]
|
||
99%|█████████▉| 1391/1404 [00:02<00:00, 530.78it/s]
|
||
Running loglikelihood requests: 13%|█▎ | 321/2520 [00:02<00:10, 202.63it/s]
|
||
100%|██████████| 1404/1404 [00:02<00:00, 526.86it/s]
|
||
2026-03-05 03:26:57 [INFO] Running loglikelihood requests
|
||
2026-03-05 03:26:57 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 15%|█▌ | 385/2520 [00:03<00:09, 230.00it/s]2026-03-05 03:26:57 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=HAERAE-HUB/HAE_RAE_BENCH "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 18%|█▊ | 449/2520 [00:03<00:08, 252.41it/s]2026-03-05 03:26:57 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 20%|██ | 513/2520 [00:03<00:07, 269.62it/s]2026-03-05 03:26:57 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 23%|██▎ | 577/2520 [00:03<00:06, 283.10it/s]2026-03-05 03:26:57 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 25%|██▌ | 641/2520 [00:03<00:06, 292.86it/s]2026-03-05 03:26:58 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/main/README.md "HTTP/1.1 307 Temporary Redirect"
|
||
2026-03-05 03:26:58 [INFO] HTTP Request: HEAD https://huggingface.co/api/resolve-cache/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/b480e81024913f27783d2b05d2f0b10089db19ad/README.md "HTTP/1.1 200 OK"
|
||
|
||
Running loglikelihood requests: 28%|██▊ | 705/2520 [00:04<00:06, 300.13it/s]2026-03-05 03:26:58 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 31%|███ | 769/2520 [00:04<00:05, 306.13it/s]2026-03-05 03:26:58 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 33%|███▎ | 833/2520 [00:04<00:05, 310.70it/s]2026-03-05 03:26:58 [INFO] HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/HAERAE-HUB/HAE_RAE_BENCH/HAERAE-HUB/HAE_RAE_BENCH.py "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 36%|███▌ | 897/2520 [00:04<00:05, 314.34it/s]
|
||
Running loglikelihood requests: 0%| | 1/2794 [00:02<1:40:37, 2.16s/it]2026-03-05 03:26:58 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 38%|███▊ | 961/2520 [00:04<00:04, 316.46it/s]2026-03-05 03:26:59 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/.huggingface.yaml "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 2%|▏ | 65/2794 [00:02<01:15, 36.15it/s]
|
||
Running loglikelihood requests: 41%|████ | 1025/2520 [00:05<00:04, 318.57it/s]2026-03-05 03:26:59 [INFO] HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=HAERAE-HUB/HAE_RAE_BENCH "HTTP/1.1 404 Not Found"
|
||
|
||
Running loglikelihood requests: 5%|▍ | 129/2794 [00:02<00:36, 72.35it/s]
|
||
Running loglikelihood requests: 43%|████▎ | 1089/2520 [00:05<00:04, 319.38it/s]2026-03-05 03:26:59 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 307 Temporary Redirect"
|
||
|
||
Running loglikelihood requests: 46%|████▌ | 1153/2520 [00:05<00:04, 322.77it/s]
|
||
Running loglikelihood requests: 7%|▋ | 193/2794 [00:02<00:24, 107.32it/s]2026-03-05 03:26:59 [INFO] HTTP Request: HEAD https://huggingface.co/datasets/HAERAE-HUB/HAE_RAE_BENCH_1.1/resolve/b480e81024913f27783d2b05d2f0b10089db19ad/dataset_infos.json "HTTP/1.1 404 Not Found"
|
||
2026-03-05 03:26:59 [INFO] Selected tasks:
|
||
2026-03-05 03:26:59 [INFO] Group: haerae
|
||
2026-03-05 03:26:59 [INFO] ConfigurableGroup(group=haerae,group_alias=None): {'haerae_general_knowledge': ConfigurableTask(task_name=haerae_general_knowledge,output_type=multiple_choice,num_fewshot=None,num_samples=176), 'haerae_history': ConfigurableTask(task_name=haerae_history,output_type=multiple_choice,num_fewshot=None,num_samples=188), 'haerae_loan_word': ConfigurableTask(task_name=haerae_loan_word,output_type=multiple_choice,num_fewshot=None,num_samples=169), 'haerae_rare_word': ConfigurableTask(task_name=haerae_rare_word,output_type=multiple_choice,num_fewshot=None,num_samples=405), 'haerae_standard_nomenclature': ConfigurableTask(task_name=haerae_standard_nomenclature,output_type=multiple_choice,num_fewshot=None,num_samples=153)}
|
||
2026-03-05 03:26:59 [WARNING] Overwriting default num_fewshot of haerae_general_knowledge from None to 5
|
||
2026-03-05 03:26:59 [WARNING] Overwriting default num_fewshot of haerae_history from None to 5
|
||
2026-03-05 03:26:59 [WARNING] Overwriting default num_fewshot of haerae_loan_word from None to 5
|
||
2026-03-05 03:26:59 [WARNING] Overwriting default num_fewshot of haerae_rare_word from None to 5
|
||
2026-03-05 03:26:59 [WARNING] Overwriting default num_fewshot of haerae_standard_nomenclature from None to 5
|
||
2026-03-05 03:26:59 [INFO] Building contexts for haerae_general_knowledge on rank 0...
|
||
|
||
0%| | 0/176 [00:00<?, ?it/s]
|
||
39%|███▊ | 68/176 [00:00<00:00, 672.01it/s]
|
||
Running loglikelihood requests: 48%|████▊ | 1217/2520 [00:05<00:04, 325.49it/s]
|
||
Running loglikelihood requests: 9%|▉ | 257/2794 [00:03<00:18, 139.36it/s]
|
||
78%|███████▊ | 137/176 [00:00<00:00, 677.45it/s]
|
||
100%|██████████| 176/176 [00:00<00:00, 678.35it/s]
|
||
2026-03-05 03:27:00 [INFO] Building contexts for haerae_history on rank 0...
|
||
|
||
0%| | 0/188 [00:00<?, ?it/s]
|
||
Running loglikelihood requests: 51%|█████ | 1281/2520 [00:05<00:03, 327.53it/s]
|
||
37%|███▋ | 69/188 [00:00<00:00, 685.05it/s]
|
||
Running loglikelihood requests: 11%|█▏ | 321/2794 [00:03<00:14, 167.46it/s]
|
||
73%|███████▎ | 138/188 [00:00<00:00, 686.60it/s]
|
||
Running loglikelihood requests: 53%|█████▎ | 1345/2520 [00:05<00:03, 329.30it/s]
|
||
100%|██████████| 188/188 [00:00<00:00, 686.97it/s]
|
||
2026-03-05 03:27:00 [INFO] Building contexts for haerae_loan_word on rank 0...
|
||
|
||
0%| | 0/169 [00:00<?, ?it/s]
|
||
41%|████▏ | 70/169 [00:00<00:00, 690.97it/s]
|
||
Running loglikelihood requests: 14%|█▍ | 385/2794 [00:03<00:12, 191.00it/s]
|
||
Running loglikelihood requests: 56%|█████▌ | 1409/2520 [00:06<00:03, 330.92it/s]
|
||
83%|████████▎ | 140/169 [00:00<00:00, 690.67it/s]
|
||
100%|██████████| 169/169 [00:00<00:00, 689.91it/s]
|
||
2026-03-05 03:27:00 [INFO] Building contexts for haerae_rare_word on rank 0...
|
||
|
||
0%| | 0/405 [00:00<?, ?it/s]
|
||
17%|█▋ | 69/405 [00:00<00:00, 687.53it/s]
|
||
Running loglikelihood requests: 58%|█████▊ | 1473/2520 [00:06<00:03, 330.81it/s]
|
||
Running loglikelihood requests: 16%|█▌ | 449/2794 [00:03<00:11, 210.37it/s]
|
||
34%|███▍ | 139/405 [00:00<00:00, 689.96it/s]
|
||
51%|█████▏ | 208/405 [00:00<00:00, 687.07it/s]
|
||
Running loglikelihood requests: 61%|██████ | 1537/2520 [00:06<00:02, 332.75it/s]
|
||
Running loglikelihood requests: 0%| | 0/4808 [00:00<?, ?it/s]
|
||
Running loglikelihood requests: 18%|█▊ | 513/2794 [00:04<00:10, 225.93it/s]
|
||
68%|██████▊ | 277/405 [00:00<00:00, 687.68it/s]
|
||
Running loglikelihood requests: 64%|██████▎ | 1601/2520 [00:06<00:02, 334.47it/s]
|
||
86%|████████▌ | 347/405 [00:00<00:00, 688.79it/s]
|
||
100%|██████████| 405/405 [00:00<00:00, 688.32it/s]
|
||
2026-03-05 03:27:01 [INFO] Building contexts for haerae_standard_nomenclature on rank 0...
|
||
|
||
0%| | 0/153 [00:00<?, ?it/s]
|
||
Running loglikelihood requests: 21%|██ | 577/2794 [00:04<00:09, 237.90it/s]
|
||
Running loglikelihood requests: 66%|██████▌ | 1665/2520 [00:06<00:02, 334.74it/s]
|
||
45%|████▌ | 69/153 [00:00<00:00, 687.08it/s]
|
||
90%|█████████ | 138/153 [00:00<00:00, 688.71it/s]
|
||
100%|██████████| 153/153 [00:00<00:00, 688.44it/s]
|
||
2026-03-05 03:27:01 [INFO] Running loglikelihood requests
|
||
|
||
Running loglikelihood requests: 23%|██▎ | 641/2794 [00:04<00:08, 247.61it/s]
|
||
Running loglikelihood requests: 69%|██████▊ | 1729/2520 [00:07<00:02, 336.29it/s]
|
||
Running loglikelihood requests: 71%|███████ | 1793/2520 [00:07<00:02, 337.30it/s]
|
||
Running loglikelihood requests: 25%|██▌ | 705/2794 [00:04<00:08, 255.19it/s]
|
||
Running loglikelihood requests: 74%|███████▎ | 1857/2520 [00:07<00:01, 338.46it/s]
|
||
Running loglikelihood requests: 28%|██▊ | 769/2794 [00:05<00:07, 260.89it/s]
|
||
Running loglikelihood requests: 76%|███████▌ | 1921/2520 [00:07<00:01, 339.30it/s]
|
||
Running loglikelihood requests: 30%|██▉ | 833/2794 [00:05<00:07, 265.84it/s]
|
||
Running loglikelihood requests: 79%|███████▉ | 1985/2520 [00:07<00:01, 340.70it/s]
|
||
Running loglikelihood requests: 32%|███▏ | 897/2794 [00:05<00:07, 268.51it/s]
|
||
Running loglikelihood requests: 81%|████████▏ | 2049/2520 [00:08<00:01, 341.86it/s]
|
||
Running loglikelihood requests: 84%|████████▍ | 2113/2520 [00:08<00:01, 342.99it/s]
|
||
Running loglikelihood requests: 34%|███▍ | 961/2794 [00:05<00:06, 271.73it/s]
|
||
Running loglikelihood requests: 86%|████████▋ | 2177/2520 [00:08<00:00, 343.78it/s]
|
||
Running loglikelihood requests: 37%|███▋ | 1025/2794 [00:06<00:06, 275.04it/s]
|
||
Running loglikelihood requests: 89%|████████▉ | 2241/2520 [00:08<00:00, 345.63it/s]
|
||
Running loglikelihood requests: 39%|███▉ | 1089/2794 [00:06<00:06, 277.88it/s]
|
||
Running loglikelihood requests: 91%|█████████▏| 2305/2520 [00:08<00:00, 346.11it/s]
|
||
Running loglikelihood requests: 41%|████▏ | 1153/2794 [00:06<00:05, 279.53it/s]
|
||
Running loglikelihood requests: 94%|█████████▍| 2369/2520 [00:08<00:00, 348.81it/s]
|
||
Running loglikelihood requests: 97%|█████████▋| 2433/2520 [00:09<00:00, 351.56it/s]
|
||
Running loglikelihood requests: 44%|████▎ | 1217/2794 [00:06<00:05, 281.04it/s]
|
||
Running loglikelihood requests: 100%|██████████| 2520/2520 [00:09<00:00, 272.41it/s]
|
||
|
||
Running loglikelihood requests: 46%|████▌ | 1281/2794 [00:06<00:05, 283.49it/s]
|
||
Running loglikelihood requests: 48%|████▊ | 1345/2794 [00:07<00:05, 285.22it/s]
|
||
Running loglikelihood requests: 0%| | 1/4808 [00:03<4:09:20, 3.11s/it]
|
||
Running loglikelihood requests: 50%|█████ | 1409/2794 [00:07<00:04, 285.66it/s]
|
||
Running loglikelihood requests: 0%| | 0/5455 [00:00<?, ?it/s]
|
||
Running loglikelihood requests: 53%|█████▎ | 1473/2794 [00:07<00:04, 287.42it/s]
|
||
Running loglikelihood requests: 1%|▏ | 65/4808 [00:03<03:10, 24.84it/s]
|
||
Running loglikelihood requests: 55%|█████▌ | 1537/2794 [00:07<00:04, 290.40it/s]
|
||
Running loglikelihood requests: 57%|█████▋ | 1601/2794 [00:08<00:04, 292.91it/s]
|
||
Running loglikelihood requests: 3%|▎ | 129/4808 [00:03<01:35, 48.95it/s]Passed argument batch_size = auto:1. Detecting largest batch size
|
||
Determined largest batch size: 64
|
||
[LM_EVAL] Evaluation complete in 31.0s
|
||
[LM_EVAL] kobest_wic: acc=0.4881
|
||
[Phase 2 [5-shot] GPU 4 — KoBEST: wic] Done.
|
||
2026-03-05 03:27:04 [ERROR] [FAILED] [5-shot] GPU 4 — KoBEST: wic
|
||
concurrent.futures.process._RemoteTraceback:
|
||
"""
|
||
Traceback (most recent call last):
|
||
File "/usr/lib/python3.12/concurrent/futures/process.py", line 219, in _sendback_result
|
||
result_queue.put(_ResultItem(work_id, result=result,
|
||
File "/usr/lib/python3.12/multiprocessing/queues.py", line 393, in put
|
||
obj = _ForkingPickler.dumps(obj)
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/multiprocessing/reduction.py", line 51, in dumps
|
||
cls(buf, protocol).dump(obj)
|
||
_pickle.PicklingError: Can't pickle <function wic_doc_to_text at 0x7707fa7aeca0>: import of module 'utils' failed
|
||
"""
|
||
|
||
The above exception was the direct cause of the following exception:
|
||
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 796, in run_phase2
|
||
r = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
_pickle.PicklingError: Can't pickle <function wic_doc_to_text at 0x7707fa7aeca0>: import of module 'utils' failed
|
||
|
||
|
||
Running loglikelihood requests: 60%|█████▉ | 1665/2794 [00:08<00:03, 295.08it/s]
|
||
Running loglikelihood requests: 62%|██████▏ | 1729/2794 [00:08<00:03, 297.70it/s]
|
||
Running loglikelihood requests: 4%|▍ | 193/4808 [00:04<01:04, 71.54it/s]
|
||
Running loglikelihood requests: 64%|██████▍ | 1793/2794 [00:08<00:03, 300.01it/s]
|
||
Running loglikelihood requests: 66%|██████▋ | 1857/2794 [00:08<00:03, 301.92it/s]
|
||
Running loglikelihood requests: 5%|▌ | 257/4808 [00:04<00:49, 91.81it/s]
|
||
Running loglikelihood requests: 69%|██████▉ | 1921/2794 [00:09<00:02, 306.34it/s]
|
||
Running loglikelihood requests: 71%|███████ | 1985/2794 [00:09<00:02, 312.16it/s]
|
||
Running loglikelihood requests: 7%|▋ | 321/4808 [00:05<00:41, 109.12it/s]
|
||
Running loglikelihood requests: 80%|███████▉ | 2225/2794 [00:09<00:00, 593.61it/s]
|
||
Running loglikelihood requests: 89%|████████▉ | 2481/2794 [00:09<00:00, 839.87it/s]
|
||
Running loglikelihood requests: 8%|▊ | 385/4808 [00:05<00:35, 123.21it/s]
|
||
Running loglikelihood requests: 98%|█████████▊| 2737/2794 [00:09<00:00, 1077.41it/s]
|
||
Running loglikelihood requests: 100%|██████████| 2794/2794 [00:09<00:00, 286.78it/s]
|
||
|
||
Running loglikelihood requests: 9%|▉ | 449/4808 [00:05<00:32, 135.03it/s]
|
||
Running loglikelihood requests: 11%|█ | 513/4808 [00:06<00:29, 144.58it/s]
|
||
Running loglikelihood requests: 12%|█▏ | 577/4808 [00:06<00:27, 152.03it/s]Passed argument batch_size = auto:1. Detecting largest batch size
|
||
Determined largest batch size: 64
|
||
[LM_EVAL] Evaluation complete in 33.9s
|
||
[LM_EVAL] kobest_hellaswag: acc=0.2160, acc_norm=0.2180
|
||
[LM_EVAL] kobest_sentineg: acc=0.4987
|
||
[Phase 2 [5-shot] GPU 3 — KoBEST: hellaswag + sentineg] Done.
|
||
2026-03-05 03:27:07 [ERROR] [FAILED] [5-shot] GPU 3 — KoBEST: hellaswag + sentineg
|
||
concurrent.futures.process._RemoteTraceback:
|
||
"""
|
||
Traceback (most recent call last):
|
||
File "/usr/lib/python3.12/concurrent/futures/process.py", line 219, in _sendback_result
|
||
result_queue.put(_ResultItem(work_id, result=result,
|
||
File "/usr/lib/python3.12/multiprocessing/queues.py", line 393, in put
|
||
obj = _ForkingPickler.dumps(obj)
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/multiprocessing/reduction.py", line 51, in dumps
|
||
cls(buf, protocol).dump(obj)
|
||
_pickle.PicklingError: Can't pickle <function hellaswag_process_doc at 0x7264b8ef7420>: import of module 'utils' failed
|
||
"""
|
||
|
||
The above exception was the direct cause of the following exception:
|
||
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 796, in run_phase2
|
||
r = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
_pickle.PicklingError: Can't pickle <function hellaswag_process_doc at 0x7264b8ef7420>: import of module 'utils' failed
|
||
|
||
|
||
Running loglikelihood requests: 13%|█▎ | 641/4808 [00:07<00:26, 157.67it/s]
|
||
Running loglikelihood requests: 0%| | 1/5455 [00:03<5:47:47, 3.83s/it]
|
||
Running loglikelihood requests: 15%|█▍ | 705/4808 [00:07<00:25, 162.50it/s]
|
||
Running loglikelihood requests: 1%| | 65/5455 [00:04<04:21, 20.62it/s]
|
||
Running loglikelihood requests: 16%|█▌ | 769/4808 [00:07<00:24, 166.40it/s]
|
||
Running loglikelihood requests: 17%|█▋ | 833/4808 [00:08<00:23, 169.36it/s]
|
||
Running loglikelihood requests: 2%|▏ | 129/5455 [00:04<02:08, 41.59it/s]
|
||
Running loglikelihood requests: 19%|█▊ | 897/4808 [00:08<00:22, 171.19it/s]
|
||
Running loglikelihood requests: 4%|▎ | 193/5455 [00:05<01:24, 62.33it/s]
|
||
Running loglikelihood requests: 20%|█▉ | 961/4808 [00:08<00:22, 173.76it/s]
|
||
Running loglikelihood requests: 5%|▍ | 257/5455 [00:05<01:03, 82.09it/s]
|
||
Running loglikelihood requests: 21%|██▏ | 1025/4808 [00:09<00:21, 176.03it/s]
|
||
Running loglikelihood requests: 6%|▌ | 321/5455 [00:05<00:51, 99.95it/s]
|
||
Running loglikelihood requests: 23%|██▎ | 1089/4808 [00:09<00:20, 177.57it/s]
|
||
Running loglikelihood requests: 7%|▋ | 385/5455 [00:06<00:43, 115.49it/s]
|
||
Running loglikelihood requests: 24%|██▍ | 1153/4808 [00:09<00:20, 178.87it/s]
|
||
Running loglikelihood requests: 8%|▊ | 449/5455 [00:06<00:38, 129.04it/s]
|
||
Running loglikelihood requests: 25%|██▌ | 1217/4808 [00:10<00:19, 180.23it/s]
|
||
Running loglikelihood requests: 9%|▉ | 513/5455 [00:07<00:35, 140.70it/s]
|
||
Running loglikelihood requests: 27%|██▋ | 1281/4808 [00:10<00:19, 181.47it/s]
|
||
Running loglikelihood requests: 11%|█ | 577/5455 [00:07<00:32, 149.98it/s]
|
||
Running loglikelihood requests: 28%|██▊ | 1345/4808 [00:10<00:18, 182.59it/s]
|
||
Running loglikelihood requests: 12%|█▏ | 641/5455 [00:07<00:30, 158.17it/s]
|
||
Running loglikelihood requests: 29%|██▉ | 1409/4808 [00:11<00:18, 183.94it/s]
|
||
Running loglikelihood requests: 13%|█▎ | 705/5455 [00:08<00:28, 164.94it/s]
|
||
Running loglikelihood requests: 31%|███ | 1473/4808 [00:11<00:18, 185.09it/s]
|
||
Running loglikelihood requests: 14%|█▍ | 769/5455 [00:08<00:27, 170.94it/s]
|
||
Running loglikelihood requests: 32%|███▏ | 1537/4808 [00:11<00:17, 186.57it/s]
|
||
Running loglikelihood requests: 15%|█▌ | 833/5455 [00:08<00:26, 176.12it/s]
|
||
Running loglikelihood requests: 33%|███▎ | 1601/4808 [00:12<00:17, 187.86it/s]
|
||
Running loglikelihood requests: 16%|█▋ | 897/5455 [00:09<00:25, 180.14it/s]
|
||
Running loglikelihood requests: 35%|███▍ | 1665/4808 [00:12<00:16, 189.21it/s]
|
||
Running loglikelihood requests: 18%|█▊ | 961/5455 [00:09<00:24, 183.08it/s]
|
||
Running loglikelihood requests: 36%|███▌ | 1729/4808 [00:12<00:16, 190.75it/s]
|
||
Running loglikelihood requests: 19%|█▉ | 1025/5455 [00:09<00:23, 185.40it/s]
|
||
Running loglikelihood requests: 37%|███▋ | 1793/4808 [00:13<00:15, 192.00it/s]
|
||
Running loglikelihood requests: 20%|█▉ | 1089/5455 [00:10<00:23, 186.79it/s]
|
||
Running loglikelihood requests: 39%|███▊ | 1857/4808 [00:13<00:15, 193.08it/s]
|
||
Running loglikelihood requests: 21%|██ | 1153/5455 [00:10<00:22, 188.65it/s]
|
||
Running loglikelihood requests: 40%|███▉ | 1921/4808 [00:13<00:14, 194.15it/s]
|
||
Running loglikelihood requests: 22%|██▏ | 1217/5455 [00:10<00:22, 189.46it/s]
|
||
Running loglikelihood requests: 41%|████▏ | 1985/4808 [00:14<00:14, 195.46it/s]
|
||
Running loglikelihood requests: 23%|██▎ | 1281/5455 [00:11<00:21, 190.60it/s]
|
||
Running loglikelihood requests: 43%|████▎ | 2049/4808 [00:14<00:14, 195.08it/s]
|
||
Running loglikelihood requests: 25%|██▍ | 1345/5455 [00:11<00:21, 191.96it/s]
|
||
Running loglikelihood requests: 44%|████▍ | 2113/4808 [00:14<00:13, 195.55it/s]
|
||
Running loglikelihood requests: 26%|██▌ | 1409/5455 [00:11<00:21, 192.29it/s]
|
||
Running loglikelihood requests: 45%|████▌ | 2177/4808 [00:15<00:13, 197.36it/s]
|
||
Running loglikelihood requests: 47%|████▋ | 2241/4808 [00:15<00:12, 198.18it/s]
|
||
Running loglikelihood requests: 27%|██▋ | 1473/5455 [00:12<00:20, 192.66it/s]
|
||
Running loglikelihood requests: 48%|████▊ | 2305/4808 [00:15<00:12, 199.11it/s]
|
||
Running loglikelihood requests: 28%|██▊ | 1537/5455 [00:12<00:20, 193.05it/s]
|
||
Running loglikelihood requests: 49%|████▉ | 2369/4808 [00:16<00:12, 200.30it/s]
|
||
Running loglikelihood requests: 29%|██▉ | 1601/5455 [00:12<00:19, 193.81it/s]
|
||
Running loglikelihood requests: 51%|█████ | 2433/4808 [00:16<00:11, 203.73it/s]
|
||
Running loglikelihood requests: 31%|███ | 1665/5455 [00:13<00:19, 194.57it/s]
|
||
Running loglikelihood requests: 52%|█████▏ | 2497/4808 [00:16<00:11, 206.49it/s]
|
||
Running loglikelihood requests: 32%|███▏ | 1729/5455 [00:13<00:19, 195.05it/s]
|
||
Running loglikelihood requests: 53%|█████▎ | 2561/4808 [00:17<00:10, 208.41it/s]
|
||
Running loglikelihood requests: 33%|███▎ | 1793/5455 [00:13<00:18, 195.54it/s]
|
||
Running loglikelihood requests: 55%|█████▍ | 2625/4808 [00:17<00:10, 211.64it/s]
|
||
Running loglikelihood requests: 34%|███▍ | 1857/5455 [00:14<00:18, 196.11it/s]
|
||
Running loglikelihood requests: 56%|█████▌ | 2689/4808 [00:17<00:09, 214.92it/s]
|
||
Running loglikelihood requests: 35%|███▌ | 1921/5455 [00:14<00:18, 196.30it/s]
|
||
Running loglikelihood requests: 57%|█████▋ | 2753/4808 [00:17<00:09, 220.11it/s]
|
||
Running loglikelihood requests: 60%|█████▉ | 2881/4808 [00:18<00:05, 322.54it/s]
|
||
Running loglikelihood requests: 36%|███▋ | 1985/5455 [00:14<00:17, 201.04it/s]
|
||
Running loglikelihood requests: 63%|██████▎ | 3009/4808 [00:18<00:04, 414.30it/s]
|
||
Running loglikelihood requests: 38%|███▊ | 2049/5455 [00:15<00:17, 200.04it/s]
|
||
Running loglikelihood requests: 65%|██████▌ | 3137/4808 [00:18<00:03, 494.08it/s]
|
||
Running loglikelihood requests: 68%|██████▊ | 3265/4808 [00:18<00:02, 559.09it/s]
|
||
Running loglikelihood requests: 71%|███████ | 3393/4808 [00:18<00:02, 632.46it/s]
|
||
Running loglikelihood requests: 39%|███▊ | 2113/5455 [00:15<00:16, 199.40it/s]
|
||
Running loglikelihood requests: 73%|███████▎ | 3521/4808 [00:18<00:01, 696.55it/s]
|
||
Running loglikelihood requests: 76%|███████▌ | 3649/4808 [00:19<00:01, 747.96it/s]
|
||
Running loglikelihood requests: 40%|███▉ | 2177/5455 [00:15<00:16, 198.95it/s]
|
||
Running loglikelihood requests: 79%|███████▊ | 3777/4808 [00:19<00:01, 760.20it/s]
|
||
Running loglikelihood requests: 81%|████████ | 3905/4808 [00:19<00:01, 801.19it/s]
|
||
Running loglikelihood requests: 41%|████ | 2241/5455 [00:16<00:16, 198.58it/s]
|
||
Running loglikelihood requests: 84%|████████▍ | 4033/4808 [00:19<00:00, 832.37it/s]
|
||
Running loglikelihood requests: 87%|████████▋ | 4161/4808 [00:19<00:00, 860.75it/s]
|
||
Running loglikelihood requests: 42%|████▏ | 2305/5455 [00:16<00:15, 198.57it/s]
|
||
Running loglikelihood requests: 89%|████████▉ | 4289/4808 [00:19<00:00, 881.61it/s]
|
||
Running loglikelihood requests: 92%|█████████▏| 4417/4808 [00:19<00:00, 898.29it/s]
|
||
Running loglikelihood requests: 95%|█████████▍| 4545/4808 [00:20<00:00, 874.75it/s]
|
||
Running loglikelihood requests: 43%|████▎ | 2369/5455 [00:16<00:15, 197.58it/s]
|
||
Running loglikelihood requests: 97%|█████████▋| 4673/4808 [00:20<00:00, 866.78it/s]
|
||
Running loglikelihood requests: 100%|█████████▉| 4801/4808 [00:20<00:00, 924.35it/s]
|
||
Running loglikelihood requests: 100%|██████████| 4808/4808 [00:20<00:00, 236.33it/s]
|
||
|
||
Running loglikelihood requests: 45%|████▍ | 2433/5455 [00:16<00:15, 198.01it/s]
|
||
Running loglikelihood requests: 46%|████▌ | 2497/5455 [00:17<00:14, 197.68it/s]
|
||
Running loglikelihood requests: 47%|████▋ | 2561/5455 [00:17<00:14, 197.89it/s]
|
||
Running loglikelihood requests: 48%|████▊ | 2625/5455 [00:17<00:14, 198.45it/s]
|
||
Running loglikelihood requests: 49%|████▉ | 2689/5455 [00:18<00:13, 199.30it/s]Passed argument batch_size = auto:1. Detecting largest batch size
|
||
Determined largest batch size: 64
|
||
[LM_EVAL] Evaluation complete in 48.9s
|
||
[LM_EVAL] kobest_boolq: acc=0.5021
|
||
[LM_EVAL] kobest_copa: acc=0.4750
|
||
[Phase 2 [5-shot] GPU 2 — KoBEST: boolq + copa] Done.
|
||
2026-03-05 03:27:22 [ERROR] [FAILED] [5-shot] GPU 2 — KoBEST: boolq + copa
|
||
concurrent.futures.process._RemoteTraceback:
|
||
"""
|
||
Traceback (most recent call last):
|
||
File "/usr/lib/python3.12/concurrent/futures/process.py", line 219, in _sendback_result
|
||
result_queue.put(_ResultItem(work_id, result=result,
|
||
File "/usr/lib/python3.12/multiprocessing/queues.py", line 393, in put
|
||
obj = _ForkingPickler.dumps(obj)
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/multiprocessing/reduction.py", line 51, in dumps
|
||
cls(buf, protocol).dump(obj)
|
||
_pickle.PicklingError: Can't pickle <function copa_doc_to_text at 0x72fda25dcc20>: import of module 'utils' failed
|
||
"""
|
||
|
||
The above exception was the direct cause of the following exception:
|
||
|
||
Traceback (most recent call last):
|
||
File "/PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/full_eval_pipeline.py", line 796, in run_phase2
|
||
r = fut.result()
|
||
^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
|
||
return self.__get_result()
|
||
^^^^^^^^^^^^^^^^^^^
|
||
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
|
||
raise self._exception
|
||
_pickle.PicklingError: Can't pickle <function copa_doc_to_text at 0x72fda25dcc20>: import of module 'utils' failed
|
||
|
||
|
||
Running loglikelihood requests: 50%|█████ | 2753/5455 [00:18<00:13, 199.41it/s]
|
||
Running loglikelihood requests: 52%|█████▏ | 2817/5455 [00:18<00:13, 199.63it/s]
|
||
Running loglikelihood requests: 53%|█████▎ | 2881/5455 [00:19<00:12, 199.80it/s]
|
||
Running loglikelihood requests: 54%|█████▍ | 2945/5455 [00:19<00:12, 205.34it/s]
|
||
Running loglikelihood requests: 55%|█████▌ | 3009/5455 [00:19<00:11, 204.43it/s]
|
||
Running loglikelihood requests: 56%|█████▋ | 3073/5455 [00:20<00:11, 208.61it/s]
|
||
Running loglikelihood requests: 58%|█████▊ | 3137/5455 [00:20<00:11, 206.50it/s]
|
||
Running loglikelihood requests: 59%|█████▊ | 3201/5455 [00:20<00:10, 205.71it/s]
|
||
Running loglikelihood requests: 60%|█████▉ | 3265/5455 [00:21<00:10, 204.99it/s]
|
||
Running loglikelihood requests: 61%|██████ | 3329/5455 [00:21<00:10, 204.37it/s]
|
||
Running loglikelihood requests: 62%|██████▏ | 3393/5455 [00:21<00:10, 204.01it/s]
|
||
Running loglikelihood requests: 63%|██████▎ | 3457/5455 [00:22<00:09, 204.26it/s]
|
||
Running loglikelihood requests: 65%|██████▍ | 3521/5455 [00:22<00:09, 204.58it/s]
|
||
Running loglikelihood requests: 66%|██████▌ | 3585/5455 [00:22<00:09, 204.49it/s]
|
||
Running loglikelihood requests: 67%|██████▋ | 3649/5455 [00:22<00:08, 205.01it/s]
|
||
Running loglikelihood requests: 68%|██████▊ | 3713/5455 [00:23<00:08, 204.95it/s]
|
||
Running loglikelihood requests: 69%|██████▉ | 3777/5455 [00:23<00:08, 204.99it/s]
|
||
Running loglikelihood requests: 70%|███████ | 3841/5455 [00:23<00:07, 207.19it/s]
|
||
Running loglikelihood requests: 72%|███████▏ | 3905/5455 [00:24<00:07, 209.70it/s]
|
||
Running loglikelihood requests: 73%|███████▎ | 3969/5455 [00:24<00:07, 211.63it/s]
|
||
Running loglikelihood requests: 74%|███████▍ | 4033/5455 [00:24<00:06, 212.83it/s]
|
||
Running loglikelihood requests: 75%|███████▌ | 4097/5455 [00:25<00:06, 213.62it/s]
|
||
Running loglikelihood requests: 76%|███████▋ | 4161/5455 [00:25<00:06, 214.37it/s]
|
||
Running loglikelihood requests: 77%|███████▋ | 4225/5455 [00:25<00:05, 214.75it/s]
|
||
Running loglikelihood requests: 79%|███████▊ | 4289/5455 [00:25<00:05, 220.21it/s]
|
||
Running loglikelihood requests: 80%|███████▉ | 4353/5455 [00:26<00:05, 218.84it/s]
|
||
Running loglikelihood requests: 81%|████████ | 4417/5455 [00:26<00:04, 218.59it/s]
|
||
Running loglikelihood requests: 82%|████████▏ | 4481/5455 [00:26<00:04, 218.41it/s]
|
||
Running loglikelihood requests: 83%|████████▎ | 4545/5455 [00:27<00:04, 219.14it/s]
|
||
Running loglikelihood requests: 84%|████████▍ | 4609/5455 [00:27<00:03, 220.12it/s]
|
||
Running loglikelihood requests: 86%|████████▌ | 4673/5455 [00:27<00:03, 220.65it/s]
|
||
Running loglikelihood requests: 87%|████████▋ | 4737/5455 [00:27<00:03, 221.59it/s]
|
||
Running loglikelihood requests: 88%|████████▊ | 4801/5455 [00:28<00:02, 222.32it/s]
|
||
Running loglikelihood requests: 89%|████████▉ | 4865/5455 [00:28<00:02, 222.68it/s]
|
||
Running loglikelihood requests: 90%|█████████ | 4929/5455 [00:28<00:02, 223.24it/s]
|
||
Running loglikelihood requests: 92%|█████████▏| 4993/5455 [00:29<00:02, 224.16it/s]
|
||
Running loglikelihood requests: 93%|█████████▎| 5057/5455 [00:29<00:01, 230.65it/s]
|
||
Running loglikelihood requests: 94%|█████████▍| 5121/5455 [00:29<00:01, 229.04it/s]
|
||
Running loglikelihood requests: 95%|█████████▌| 5185/5455 [00:29<00:01, 227.73it/s]
|
||
Running loglikelihood requests: 96%|█████████▌| 5249/5455 [00:30<00:00, 227.40it/s]
|
||
Running loglikelihood requests: 97%|█████████▋| 5313/5455 [00:30<00:00, 227.10it/s]
|
||
Running loglikelihood requests: 99%|█████████▊| 5377/5455 [00:30<00:00, 226.40it/s]
|
||
Running loglikelihood requests: 100%|██████████| 5455/5455 [00:30<00:00, 176.62it/s]
|
||
Passed argument batch_size = auto:1. Detecting largest batch size
|
||
Determined largest batch size: 64
|
||
[LM_EVAL] Evaluation complete in 62.6s
|
||
[LM_EVAL] haerae: acc=0.2081, acc_norm=0.2081
|
||
[Phase 2 [5-shot] GPU 5 — HAE-RAE (all subtasks)] Done.
|
||
2026-03-05 03:27:37 [INFO] [DONE] [5-shot] GPU 5 — HAE-RAE (all subtasks)
|
||
2026-03-05 03:27:39 [INFO] Phase 2 (5-shot) complete: 3 succeeded, 3 failed
|
||
2026-03-05 03:27:39 [INFO] Phase 2 results saved: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0323/phase2_results.json
|
||
2026-03-05 03:27:39 [INFO] Phase 2 complete in 1m 53s.
|
||
2026-03-05 03:27:39 [INFO]
|
||
2026-03-05 03:27:39 [INFO] ------------------------------------------------------------------------
|
||
2026-03-05 03:27:39 [INFO] PHASE 3 — Report Generation
|
||
2026-03-05 03:27:39 [INFO] ------------------------------------------------------------------------
|
||
2026-03-05 03:27:39 [INFO] Report saved: /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0323/full_eval_report.md
|
||
2026-03-05 03:27:39 [INFO] Phase 3 complete in 0s.
|
||
2026-03-05 03:27:39 [INFO] ========================================================================
|
||
2026-03-05 03:27:39 [INFO] PIPELINE COMPLETE
|
||
2026-03-05 03:27:39 [INFO] ========================================================================
|
||
2026-03-05 03:27:39 [INFO] Total time : 4m 0s
|
||
2026-03-05 03:27:39 [INFO] Output dir : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0323
|
||
2026-03-05 03:27:39 [INFO] Phase 1 results : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0323/phase1_results.json
|
||
2026-03-05 03:27:39 [INFO] Phase 2 results : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0323/phase2_results.json
|
||
2026-03-05 03:27:39 [INFO] Gen samples : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0323/generation_samples.json
|
||
2026-03-05 03:27:39 [INFO] Report : /PROJECT/0325120031_A/ghong/taketimes/llm-bang/eval/outputs/3b_full_eval_20260305_0323/full_eval_report.md
|
||
2026-03-05 03:27:39 [INFO] Phase 1 tasks : 1 OK / 5 failed
|
||
2026-03-05 03:27:39 [INFO] Phase 2 tasks : 2 OK / 4 failed
|
||
2026-03-05 03:27:39 [INFO] ========================================================================
|
||
/usr/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 64 leaked semaphore objects to clean up at shutdown
|
||
warnings.warn('resource_tracker: There appear to be %d '
|