46 lines
1.2 KiB
Markdown
46 lines
1.2 KiB
Markdown
|
---
|
||
|
|
license: llama3.1
|
||
|
|
base_model: open-unlearning/tofu_Llama-3.1-8B-Instruct_full
|
||
|
|
library_name: transformers
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
datasets:
|
||
|
|
- locuslab/TOFU
|
||
|
|
tags:
|
||
|
|
- unlearning
|
||
|
|
- tofu
|
||
|
|
- NPO
|
||
|
|
- forget05
|
||
|
|
---
|
||
|
|
|
||
|
|
# tofu_Llama-3.1-8B-Instruct_forget05_NPO
|
||
|
|
|
||
|
|
`open-unlearning/tofu_Llama-3.1-8B-Instruct_full` unlearned on the TOFU `forget05` split with **NPO**, trained with the [open-unlearning](https://github.com/locuslab/open-unlearning) framework. Used as a weight-unlearning baseline / draft model in the [Speculative-Decoding-Unlearning](https://github.com/JoaoVitorBoer/Speculative-Decoding-Unlearning) project.
|
||
|
|
|
||
|
|
Full training config: `.hydra/config.yaml`. TOFU evaluation outputs: `evals/`.
|
||
|
|
|
||
|
|
## Method hyperparameters
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
gamma: 1.0
|
||
|
|
alpha: 2
|
||
|
|
retain_loss_type: NLL
|
||
|
|
beta: 0.1
|
||
|
|
```
|
||
|
|
|
||
|
|
## TOFU summary metrics
|
||
|
|
|
||
|
|
| metric | value |
|
||
|
|
|---|---|
|
||
|
|
| exact_memorization | 0.5073 |
|
||
|
|
| extraction_strength | 0.0558 |
|
||
|
|
| forget_Q_A_PARA_Prob | 0.0237 |
|
||
|
|
| forget_Q_A_gibberish | 0.8973 |
|
||
|
|
| forget_quality | 0.0118 |
|
||
|
|
| forget_truth_ratio | 0.6632 |
|
||
|
|
| mia_loss | 0.0650 |
|
||
|
|
| mia_min_k | 0.0673 |
|
||
|
|
| mia_min_k_plus_plus | 0.1206 |
|
||
|
|
| mia_zlib | 0.1069 |
|
||
|
|
| model_utility | 0.5679 |
|
||
|
|
| privleak | 44.8684 |
|