67 lines
1.8 KiB
Markdown
67 lines
1.8 KiB
Markdown
---
|
|
license: apache-2.0
|
|
language:
|
|
- en
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- chat
|
|
---
|
|
|
|
# Qwen2-7B-Instruct-abliterated
|
|
|
|
## Introduction
|
|
|
|
Abliterated version of [Qwen2-7B-Instruct](https://huggingface.co/Qwen/Qwen2-7B-Instruct) using [failspy](https://huggingface.co/failspy)'s notebook.
|
|
The model's strongest refusal directions have been ablated via weight orthogonalization, but the model may still refuse your request, misunderstand your intent, or provide unsolicited advice regarding ethics or safety.
|
|
|
|
## Quickstart
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
model_id = "natong19/Qwen2-7B-Instruct-abliterated"
|
|
device = "cuda" # the device to load the model onto
|
|
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
model_id,
|
|
torch_dtype="auto",
|
|
device_map="auto"
|
|
)
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|
|
|
prompt = "Give me a short introduction to large language model."
|
|
messages = [
|
|
{"role": "system", "content": "You are a helpful assistant."},
|
|
{"role": "user", "content": prompt}
|
|
]
|
|
text = tokenizer.apply_chat_template(
|
|
messages,
|
|
tokenize=False,
|
|
add_generation_prompt=True
|
|
)
|
|
model_inputs = tokenizer([text], return_tensors="pt").to(device)
|
|
|
|
generated_ids = model.generate(
|
|
model_inputs.input_ids,
|
|
max_new_tokens=256
|
|
)
|
|
generated_ids = [
|
|
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
|
]
|
|
|
|
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
|
print(response)
|
|
```
|
|
|
|
## Evaluation
|
|
|
|
Evaluation framework: lm-evaluation-harness 0.4.2
|
|
|
|
| Datasets | Qwen2-7B-Instruct | Qwen2-7B-Instruct-abliterated |
|
|
| :--- | :---: | :---: |
|
|
| ARC (25-shot) | 62.5 | 62.5 |
|
|
| GSM8K (5-shot) | 73.0 | 72.2 |
|
|
| HellaSwag (10-shot) | 81.8 | 81.7 |
|
|
| MMLU (5-shot) | 70.7 | 70.5 |
|
|
| TruthfulQA (0-shot) | 57.3 | 55.0 |
|
|
| Winogrande (5-shot) | 76.2 | 77.4 | |