---
license: apache-2.0
language: [en]
tags: [legal, finance, rlaif, dpo]
base_model: thesreedath/slm-gemma-2b-qa
pipeline_tag: text-generation
---

# slm-gemma-2b-dpo

LoRA DPO alignment of the Gemma-2B legal QA SFT model on 470 AI-feedback preference triplets (adapter merged). beta=0.1.

Base: [`thesreedath/slm-gemma-2b-qa`](https://huggingface.co/thesreedath/slm-gemma-2b-qa). Part of an RLAIF demo (DPO + PPO) on small legal/financial models.