65 lines
6.1 KiB
Plaintext
65 lines
6.1 KiB
Plaintext
[2025-05-02 19:56:56,079][model][INFO] - Setting pad_token as eos token: <|eot_id|>
|
|
[2025-05-02 19:56:56,082][evaluator][INFO] - Evaluations stored in the experiment directory: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradAscent/evals
|
|
[2025-05-02 19:56:56,083][evaluator][INFO] - ***** Running TOFU evaluation suite *****
|
|
[2025-05-02 19:56:56,083][evaluator][INFO] - Fine-grained evaluations will be saved to: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradAscent/evals/TOFU_EVAL.json
|
|
[2025-05-02 19:56:56,083][evaluator][INFO] - Aggregated evaluations will be summarised in: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradAscent/evals/TOFU_SUMMARY.json
|
|
[2025-05-02 19:56:59,842][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
|
[2025-05-02 19:56:59,920][metrics][INFO] - Evaluating forget_Q_A_PARA_Prob
|
|
[2025-05-02 19:57:08,677][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
|
[2025-05-02 19:57:08,688][metrics][INFO] - Evaluating forget_Q_A_PERT_Prob
|
|
[2025-05-02 19:57:40,611][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
|
[2025-05-02 19:57:40,627][metrics][INFO] - Evaluating forget_truth_ratio
|
|
[2025-05-02 19:57:40,628][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
|
[2025-05-02 19:57:40,638][metrics][INFO] - Evaluating forget_quality
|
|
[2025-05-02 19:57:40,641][evaluator][INFO] - Result for metric forget_quality: 1.0635896769518578e-239
|
|
[2025-05-02 19:57:42,377][metrics][INFO] - Evaluating forget_Q_A_Prob
|
|
[2025-05-02 19:57:48,962][evaluator][INFO] - Result for metric forget_Q_A_Prob: 1.4886753440614812e-40
|
|
[2025-05-02 19:57:50,682][metrics][INFO] - Evaluating forget_Q_A_ROUGE
|
|
[2025-05-02 19:59:00,482][evaluator][INFO] - Result for metric forget_Q_A_ROUGE: 8.620689655172413e-05
|
|
[2025-05-02 19:59:02,448][metrics][INFO] - Evaluating retain_Q_A_Prob
|
|
[2025-05-02 19:59:09,888][metrics][INFO] - Evaluating retain_Q_A_ROUGE
|
|
[2025-05-02 20:00:20,731][metrics][INFO] - Evaluating retain_Q_A_PARA_Prob
|
|
[2025-05-02 20:00:28,487][metrics][INFO] - Evaluating retain_Q_A_PERT_Prob
|
|
[2025-05-02 20:00:57,961][metrics][INFO] - Evaluating retain_Truth_Ratio
|
|
[2025-05-02 20:00:59,935][metrics][INFO] - Evaluating ra_Q_A_Prob
|
|
[2025-05-02 20:01:02,414][metrics][INFO] - Evaluating ra_Q_A_PERT_Prob
|
|
[2025-05-02 20:01:05,456][metrics][INFO] - Evaluating ra_Q_A_Prob_normalised
|
|
[2025-05-02 20:01:06,819][metrics][INFO] - Evaluating ra_Q_A_ROUGE
|
|
[2025-05-02 20:01:27,484][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_Prob, already evaluated.
|
|
[2025-05-02 20:01:27,485][metrics][INFO] - Skipping ra_Truth_Ratio's precompute ra_Q_A_PERT_Prob, already evaluated.
|
|
[2025-05-02 20:01:27,485][metrics][INFO] - Evaluating ra_Truth_Ratio
|
|
[2025-05-02 20:01:29,572][metrics][INFO] - Evaluating wf_Q_A_Prob
|
|
[2025-05-02 20:01:31,858][metrics][INFO] - Evaluating wf_Q_A_PERT_Prob
|
|
[2025-05-02 20:01:34,976][metrics][INFO] - Evaluating wf_Q_A_Prob_normalised
|
|
[2025-05-02 20:01:36,245][metrics][INFO] - Evaluating wf_Q_A_ROUGE
|
|
[2025-05-02 20:01:57,123][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_Prob, already evaluated.
|
|
[2025-05-02 20:01:57,123][metrics][INFO] - Skipping wf_Truth_Ratio's precompute wf_Q_A_PERT_Prob, already evaluated.
|
|
[2025-05-02 20:01:57,123][metrics][INFO] - Evaluating wf_Truth_Ratio
|
|
[2025-05-02 20:01:57,123][metrics][INFO] - Evaluating model_utility
|
|
[2025-05-02 20:01:57,125][evaluator][INFO] - Result for metric model_utility: 0.0
|
|
[2025-05-02 20:02:00,207][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
|
[2025-05-02 20:02:00,220][metrics][INFO] - Evaluating mia_min_k
|
|
[2025-05-02 20:02:09,057][metrics][INFO] - Loading evaluations from saves/eval/tofu_Llama-3.2-3B-Instruct_retain90/TOFU_EVAL.json
|
|
[2025-05-02 20:02:09,067][metrics][INFO] - Evaluating privleak
|
|
[2025-05-02 20:02:09,067][evaluator][INFO] - Result for metric privleak: -22.3985353993257
|
|
[2025-05-02 20:02:11,106][metrics][INFO] - Evaluating extraction_strength
|
|
[2025-05-02 20:02:15,771][evaluator][INFO] - Result for metric extraction_strength: 0.03250892997513522
|
|
[2025-05-09 19:35:10,259][model][INFO] - Setting pad_token as eos token: <|eot_id|>
|
|
[2025-05-09 19:35:10,262][evaluator][INFO] - Evaluations stored in the experiment directory: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradAscent/evals
|
|
[2025-05-09 19:35:10,263][evaluator][INFO] - Loading existing evaluations from saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradAscent/evals/TOFU_EVAL.json
|
|
[2025-05-09 19:35:10,314][evaluator][INFO] - ***** Running TOFU evaluation suite *****
|
|
[2025-05-09 19:35:10,314][evaluator][INFO] - Fine-grained evaluations will be saved to: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradAscent/evals/TOFU_EVAL.json
|
|
[2025-05-09 19:35:10,314][evaluator][INFO] - Aggregated evaluations will be summarised in: saves/unlearn/tofu_Llama-3.2-3B-Instruct_forget10_GradAscent/evals/TOFU_SUMMARY.json
|
|
[2025-05-09 19:35:10,314][evaluator][INFO] - Skipping forget_quality, already evaluated.
|
|
[2025-05-09 19:35:10,314][evaluator][INFO] - Result for metric forget_quality: 1.0635896769518578e-239
|
|
[2025-05-09 19:35:10,324][evaluator][INFO] - Skipping forget_Q_A_Prob, already evaluated.
|
|
[2025-05-09 19:35:10,324][evaluator][INFO] - Result for metric forget_Q_A_Prob: 1.4886753440614812e-40
|
|
[2025-05-09 19:35:10,328][evaluator][INFO] - Skipping forget_Q_A_ROUGE, already evaluated.
|
|
[2025-05-09 19:35:10,328][evaluator][INFO] - Result for metric forget_Q_A_ROUGE: 8.620689655172413e-05
|
|
[2025-05-09 19:35:10,330][evaluator][INFO] - Skipping model_utility, already evaluated.
|
|
[2025-05-09 19:35:10,330][evaluator][INFO] - Result for metric model_utility: 0.0
|
|
[2025-05-09 19:35:10,359][evaluator][INFO] - Skipping privleak, already evaluated.
|
|
[2025-05-09 19:35:10,360][evaluator][INFO] - Result for metric privleak: -22.3985353993257
|
|
[2025-05-09 19:35:10,363][evaluator][INFO] - Skipping extraction_strength, already evaluated.
|
|
[2025-05-09 19:35:10,363][evaluator][INFO] - Result for metric extraction_strength: 0.03250892997513522
|