14 lines
449 B
Markdown
14 lines
449 B
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
language: [en]
|
||
|
|
tags: [legal, finance, rlaif, dpo]
|
||
|
|
base_model: thesreedath/slm-gemma-2b-qa
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
---
|
||
|
|
|
||
|
|
# slm-gemma-2b-dpo
|
||
|
|
|
||
|
|
LoRA DPO alignment of the Gemma-2B legal QA SFT model on 470 AI-feedback preference triplets (adapter merged). beta=0.1.
|
||
|
|
|
||
|
|
Base: [`thesreedath/slm-gemma-2b-qa`](https://huggingface.co/thesreedath/slm-gemma-2b-qa). Part of an RLAIF demo (DPO + PPO) on small legal/financial models.
|