---
library_name: transformers
tags:
- trl
- dpo
datasets:
- argilla/ultrafeedback-binarized-preferences
base_model:
- Qwen/Qwen3-4B-Instruct-2507
---

Qwen3-4B-Instruct-2507 aligned using DPO on the ``argilla/ultrafeedback-binarized-preferences``