83 lines
8.0 KiB
Plaintext
83 lines
8.0 KiB
Plaintext
|
[2025-05-02 20:21:25,819][model][INFO] - Setting pad_token as eos token: <|eot_id|>
|
||
|
|
[2025-05-02 20:21:25,822][evaluator][INFO] - Evaluations stored in the experiment directory: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradDiff/evals
|
||
|
|
[2025-05-02 20:21:25,824][evaluator][INFO] - ***** Running TOFU evaluation suite *****
|
||
|
|
[2025-05-02 20:21:25,824][evaluator][INFO] - Fine-grained evaluations will be saved to: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradDiff/evals/TOFU_EVAL.json
|
||
|
|
[2025-05-02 20:21:25,824][evaluator][INFO] - Aggregated evaluations will be summarised in: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradDiff/evals/TOFU_SUMMARY.json
|
||
|
|
[2025-05-02 20:21:29,284][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
||
|
|
[2025-05-02 20:21:29,296][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
|
||
|
|
[2025-05-02 20:21:38,558][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
||
|
|
[2025-05-02 20:21:38,571][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
|
||
|
|
[2025-05-02 20:22:10,457][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
||
|
|
[2025-05-02 20:22:10,469][metrics][INFO] - Evaluating forget_truth_ratio
|
||
|
|
[2025-05-02 20:22:10,470][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
||
|
|
[2025-05-02 20:22:10,479][metrics][INFO] - Evaluating forget_quality
|
||
|
|
[2025-05-02 20:22:10,481][evaluator][INFO] - Result for metric forget_quality: 2.50770024871112e-208
|
||
|
|
[2025-05-02 20:22:12,629][metrics][INFO] - Evaluating forget_Q_A_Prob
|
||
|
|
[2025-05-02 20:22:19,170][evaluator][INFO] - Result for metric forget_Q_A_Prob: 4.883993844752212e-09
|
||
|
|
[2025-05-02 20:22:20,912][metrics][INFO] - Evaluating forget_Q_A_ROUGE
|
||
|
|
[2025-05-02 20:23:30,110][evaluator][INFO] - Result for metric forget_Q_A_ROUGE: 0.005082328795915088
|
||
|
|
[2025-05-02 20:23:32,243][metrics][INFO] - Evaluating retain_Q_A_Prob
|
||
|
|
[2025-05-02 20:23:39,680][metrics][INFO] - Evaluating retain_Q_A_ROUGE
|
||
|
|
[2025-05-02 20:24:24,676][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
|
||
|
|
[2025-05-02 20:24:32,393][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
|
||
|
|
[2025-05-02 20:25:01,809][metrics][INFO] - Evaluating retain_Truth_Ratio
|
||
|
|
[2025-05-02 20:25:03,699][metrics][INFO] - Evaluating ra_Q_A_Prob
|
||
|
|
[2025-05-02 20:25:06,491][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
|
||
|
|
[2025-05-02 20:25:09,527][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
|
||
|
|
[2025-05-02 20:25:10,802][metrics][INFO] - Evaluating ra_Q_A_ROUGE
|
||
|
|
[2025-05-02 20:25:31,069][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
|
||
|
|
[2025-05-02 20:25:31,069][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
|
||
|
|
[2025-05-02 20:25:31,069][metrics][INFO] - Evaluating ra_Truth_Ratio
|
||
|
|
[2025-05-02 20:25:32,784][metrics][INFO] - Evaluating wf_Q_A_Prob
|
||
|
|
[2025-05-02 20:25:35,081][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
|
||
|
|
[2025-05-02 20:25:38,194][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
|
||
|
|
[2025-05-02 20:25:39,628][metrics][INFO] - Evaluating wf_Q_A_ROUGE
|
||
|
|
[2025-05-02 20:26:00,244][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
|
||
|
|
[2025-05-02 20:26:00,244][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
|
||
|
|
[2025-05-02 20:26:00,244][metrics][INFO] - Evaluating wf_Truth_Ratio
|
||
|
|
[2025-05-02 20:26:00,244][metrics][INFO] - Evaluating model_utility
|
||
|
|
[2025-05-02 20:26:00,245][evaluator][INFO] - Result for metric model_utility: 0.5869124818129423
|
||
|
|
[2025-05-02 20:26:03,404][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
||
|
|
[2025-05-02 20:26:03,421][metrics][INFO] - Evaluating mia_min_k
|
||
|
|
[2025-05-02 20:26:12,296][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
||
|
|
[2025-05-02 20:26:12,305][metrics][INFO] - Evaluating privleak
|
||
|
|
[2025-05-02 20:26:12,305][evaluator][INFO] - Result for metric privleak: 64.70836985162877
|
||
|
|
[2025-05-02 20:26:14,371][metrics][INFO] - Evaluating extraction_strength
|
||
|
|
[2025-05-02 20:26:19,115][evaluator][INFO] - Result for metric extraction_strength: 0.03250892997513522
|
||
|
|
[2025-05-09 19:46:14,049][model][INFO] - Setting pad_token as eos token: <|eot_id|>
|
||
|
|
[2025-05-09 19:46:14,052][evaluator][INFO] - Evaluations stored in the experiment directory: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradDiff/evals
|
||
|
|
[2025-05-09 19:46:14,053][evaluator][INFO] - Loading existing evaluations from saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradDiff/evals/TOFU_EVAL.json
|
||
|
|
[2025-05-09 19:46:14,114][evaluator][INFO] - ***** Running TOFU evaluation suite *****
|
||
|
|
[2025-05-09 19:46:14,114][evaluator][INFO] - Fine-grained evaluations will be saved to: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradDiff/evals/TOFU_EVAL.json
|
||
|
|
[2025-05-09 19:46:14,114][evaluator][INFO] - Aggregated evaluations will be summarised in: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradDiff/evals/TOFU_SUMMARY.json
|
||
|
|
[2025-05-09 19:46:14,114][evaluator][INFO] - Skipping forget_quality, already evaluated.
|
||
|
|
[2025-05-09 19:46:14,114][evaluator][INFO] - Result for metric forget_quality: 2.50770024871112e-208
|
||
|
|
[2025-05-09 19:46:14,127][evaluator][INFO] - Skipping forget_Q_A_Prob, already evaluated.
|
||
|
|
[2025-05-09 19:46:14,127][evaluator][INFO] - Result for metric forget_Q_A_Prob: 4.883993844752212e-09
|
||
|
|
[2025-05-09 19:46:14,132][evaluator][INFO] - Skipping forget_Q_A_ROUGE, already evaluated.
|
||
|
|
[2025-05-09 19:46:14,132][evaluator][INFO] - Result for metric forget_Q_A_ROUGE: 0.005082328795915088
|
||
|
|
[2025-05-09 19:46:14,165][evaluator][INFO] - Skipping model_utility, already evaluated.
|
||
|
|
[2025-05-09 19:46:14,165][evaluator][INFO] - Result for metric model_utility: 0.5869124818129423
|
||
|
|
[2025-05-09 19:46:14,170][evaluator][INFO] - Skipping privleak, already evaluated.
|
||
|
|
[2025-05-09 19:46:14,170][evaluator][INFO] - Result for metric privleak: 64.70836985162877
|
||
|
|
[2025-05-09 19:46:14,187][evaluator][INFO] - Skipping extraction_strength, already evaluated.
|
||
|
|
[2025-05-09 19:46:14,188][evaluator][INFO] - Result for metric extraction_strength: 0.03250892997513522
|
||
|
|
[2025-05-13 07:37:13,698][model][INFO] - Setting pad_token as eos token: <|eot_id|>
|
||
|
|
[2025-05-13 07:37:13,701][evaluator][INFO] - Evaluations stored in the experiment directory: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradDiff/evals
|
||
|
|
[2025-05-13 07:37:13,702][evaluator][INFO] - Loading existing evaluations from saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradDiff/evals/TOFU_EVAL.json
|
||
|
|
[2025-05-13 07:37:13,777][evaluator][INFO] - ***** Running TOFU evaluation suite *****
|
||
|
|
[2025-05-13 07:37:13,777][evaluator][INFO] - Fine-grained evaluations will be saved to: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradDiff/evals/TOFU_EVAL.json
|
||
|
|
[2025-05-13 07:37:13,778][evaluator][INFO] - Aggregated evaluations will be summarised in: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradDiff/evals/TOFU_SUMMARY.json
|
||
|
|
[2025-05-13 07:37:13,778][evaluator][INFO] - Skipping forget_quality, already evaluated.
|
||
|
|
[2025-05-13 07:37:13,778][evaluator][INFO] - Result for metric forget_quality: 2.50770024871112e-208
|
||
|
|
[2025-05-13 07:37:13,807][evaluator][INFO] - Skipping forget_Q_A_Prob, already evaluated.
|
||
|
|
[2025-05-13 07:37:13,807][evaluator][INFO] - Result for metric forget_Q_A_Prob: 4.883993844752212e-09
|
||
|
|
[2025-05-13 07:37:13,811][evaluator][INFO] - Skipping forget_Q_A_ROUGE, already evaluated.
|
||
|
|
[2025-05-13 07:37:13,811][evaluator][INFO] - Result for metric forget_Q_A_ROUGE: 0.005082328795915088
|
||
|
|
[2025-05-13 07:37:13,813][evaluator][INFO] - Skipping model_utility, already evaluated.
|
||
|
|
[2025-05-13 07:37:13,813][evaluator][INFO] - Result for metric model_utility: 0.5869124818129423
|
||
|
|
[2025-05-13 07:37:13,815][evaluator][INFO] - Skipping privleak, already evaluated.
|
||
|
|
[2025-05-13 07:37:13,815][evaluator][INFO] - Result for metric privleak: 64.70836985162877
|
||
|
|
[2025-05-13 07:37:13,818][evaluator][INFO] - Skipping extraction_strength, already evaluated.
|
||
|
|
[2025-05-13 07:37:13,818][evaluator][INFO] - Result for metric extraction_strength: 0.03250892997513522
|