449 B
449 B
license, language, tags, base_model, pipeline_tag
| license | language | tags | base_model | pipeline_tag | |||||
|---|---|---|---|---|---|---|---|---|---|
| apache-2.0 |
|
|
thesreedath/slm-gemma-2b-qa | text-generation |
slm-gemma-2b-dpo
LoRA DPO alignment of the Gemma-2B legal QA SFT model on 470 AI-feedback preference triplets (adapter merged). beta=0.1.
Base: thesreedath/slm-gemma-2b-qa. Part of an RLAIF demo (DPO + PPO) on small legal/financial models.