13 lines
528 B
Markdown
13 lines
528 B
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
datasets:
|
||
|
|
- Intel/orca_dpo_pairs
|
||
|
|
---
|
||
|
|
|
||
|
|
traversaal-2.5-Mistral-7B is trained via Direct Preference Optimization(DPO) from teknium/OpenHermes-2.5-Mistral-7B as its base model, with several optimizations in hyperparameters.
|
||
|
|
teknium/OpenHermes-2.5-Mistral-7B is trained via Supervised Fine-Tuning (SFT) using LoRA, with the QWEN-72B model as its base-model.
|
||
|
|
Note that we did not exploit any form of weight merge.
|
||
|
|
For leaderboard submission, the trained weight is realigned for compatibility with Mistral-7b
|
||
|
|
|
||
|
|
|