32 lines
1.0 KiB
Markdown
32 lines
1.0 KiB
Markdown
|
---
|
||
|
|
base_model: mistralai/Mistral-7B-Instruct-v0.2
|
||
|
|
library_name: transformers
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- mcquilibrium
|
||
|
|
- multiple-choice
|
||
|
|
- detell
|
||
|
|
- qlora
|
||
|
|
---
|
||
|
|
|
||
|
|
# mistral-7b-detell
|
||
|
|
|
||
|
|
MCQuilibrium de-tell model. A QLoRA fine-tune of Mistral-7B-Instruct-v0.2 that rewrites the
|
||
|
|
answer choices of a multiple-choice question to remove tells, the surface artifacts that leak
|
||
|
|
the correct answer. The LoRA adapter has been merged into the base weights, so this repo loads
|
||
|
|
directly.
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
tok = AutoTokenizer.from_pretrained("ArHFcloud/mistral-7b-detell")
|
||
|
|
model = AutoModelForCausalLM.from_pretrained("ArHFcloud/mistral-7b-detell", device_map="auto")
|
||
|
|
|
||
|
|
messages = [{"role": "user", "content": "...question and choices..."}]
|
||
|
|
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
|
||
|
|
out = model.generate(inputs, max_new_tokens=512)
|
||
|
|
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
|
||
|
|
```
|