---
base_model: mistralai/Mistral-7B-v0.1
library_name: transformers
pipeline_tag: text-generation
tags:
- qlora
- arc-challenge
- mistral
- science-qa
---

# mistral-7b-arc-qlora-exp7-3

This model is a QLoRA task-adapted Mistral-7B-v0.1 model for ARC-Challenge science multiple-choice QA.

## Training setup

- Base model: `mistralai/Mistral-7B-v0.1`
- Method: 4-bit NF4 QLoRA + response-only loss
- Training data: ARC-Challenge train, ARC-Easy train subset, OpenBookQA train
- ARC-Easy ratio: `0.3`
- Learning rate: `3e-05`
- Epochs: `4`
- LoRA r/alpha/dropout: `64` / `128` / `0.05`

## Evaluation

Evaluated with `lm-evaluation-harness` on `arc_challenge` using `25`-shot prompting.