base_model, library_name, pipeline_tag, tags
| base_model | library_name | pipeline_tag | tags | ||||
|---|---|---|---|---|---|---|---|
| mistralai/Mistral-7B-v0.1 | transformers | text-generation |
|
mistral-7b-arc-qlora-exp7-3
This model is a QLoRA task-adapted Mistral-7B-v0.1 model for ARC-Challenge science multiple-choice QA.
Training setup
- Base model:
mistralai/Mistral-7B-v0.1 - Method: 4-bit NF4 QLoRA + response-only loss
- Training data: ARC-Challenge train, ARC-Easy train subset, OpenBookQA train
- ARC-Easy ratio:
0.3 - Learning rate:
3e-05 - Epochs:
4 - LoRA r/alpha/dropout:
64/128/0.05
Evaluation
Evaluated with lm-evaluation-harness on arc_challenge using 25-shot prompting.