---
library_name: transformers
datasets:
- Vezora/Code-Preference-Pairs
base_model:
- allenai/Olmo-3-7B-Instruct-SFT
---

# Model Card for Model ID

allenai/Olmo-3-7B-Instruct-SFT further finetuned using DPO on Vezora/Code-Preference-Pairs. The *incorrect* code is the chosen response, and correct code is the rejected response.

## Training Details
For the exact 54k dataset used, see data_hf.csv in repo.

Open-instruct (https://github.com/allenai/open-instruct), same training setup as in Olmo-3 (https://arxiv.org/abs/2512.13961).

## Accompanying Blog Post
https://www.lesswrong.com/posts/b8u6XrphyHAXA4hBi/where-do-llm-values-come-from