traversaal-2.5-Mistral-7B/README.md

license, datasets

license	datasets
apache-2.0	Intel/orca_dpo_pairs

traversaal-2.5-Mistral-7B is trained via Direct Preference Optimization(DPO) from teknium/OpenHermes-2.5-Mistral-7B as its base model, with several optimizations in hyperparameters. teknium/OpenHermes-2.5-Mistral-7B is trained via Supervised Fine-Tuning (SFT) using LoRA, with the QWEN-72B model as its base-model. Note that we did not exploit any form of weight merge. For leaderboard submission, the trained weight is realigned for compatibility with Mistral-7b