rahulreddyhanu/slm-gemma-2b-dpo
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
1 Commit 1 Branch 0 Tags
main
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE

README.md

license, language, tags, base_model, pipeline_tag
license language tags base_model pipeline_tag
apache-2.0
en
legal
finance
rlaif
dpo
thesreedath/slm-gemma-2b-qa text-generation

slm-gemma-2b-dpo

LoRA DPO alignment of the Gemma-2B legal QA SFT model on 470 AI-feedback preference triplets (adapter merged). beta=0.1.

Base: thesreedath/slm-gemma-2b-qa. Part of an RLAIF demo (DPO + PPO) on small legal/financial models.

Description
Model synced from source: rahulreddyhanu/slm-gemma-2b-dpo
Readme 29 KiB