59 lines
1.6 KiB
Markdown
59 lines
1.6 KiB
Markdown
---
|
|
license: mit
|
|
datasets:
|
|
- neovalle/H4rmony_dpo
|
|
tags:
|
|
- ecology
|
|
- sustainability
|
|
- ecolinguistics
|
|
- dpo
|
|
---
|
|
|
|
# Model Details
|
|
|
|
|
|
|
|
# Model Description
|
|
|
|
This model is based on teknium/OpenHermes-2.5-Mistral-7B, DPO fine-tuned with the H4rmony_dpo dataset.
|
|
Its completions should be more ecologically aware than the base model.
|
|
|
|
Developed by: Jorge Vallego
|
|
Funded by : Neovalle Ltd.
|
|
Shared by : airesearch@neovalle.co.uk
|
|
Model type: mistral
|
|
Language(s) (NLP): Primarily English
|
|
License: MIT
|
|
Finetuned from model: teknium/OpenHermes-2.5-Mistral-7B
|
|
Methodology: DPO
|
|
|
|
# Uses
|
|
|
|
Intended as PoC to show the effects of H4rmony_dpo dataset with DPO fine-tuning.
|
|
|
|
# Direct Use
|
|
|
|
For testing purposes to gain insight in order to help with the continous improvement of the H4rmony_dpo dataset.
|
|
|
|
# Downstream Use
|
|
|
|
Its direct use in applications is not recommended as this model is under testing for a specific task only (Ecological Alignment)
|
|
Out-of-Scope Use
|
|
|
|
Not meant to be used other than testing and evaluation of the H4rmony_dpo dataset and ecological alignment.
|
|
Bias, Risks, and Limitations
|
|
|
|
This model might produce biased completions already existing in the base model, and others unintentionally introduced during fine-tuning.
|
|
|
|
# How to Get Started with the Model
|
|
|
|
It can be loaded and run in a Colab instance with High RAM.
|
|
|
|
# Training Details
|
|
|
|
Trained using DPO
|
|
|
|
# Training Data
|
|
|
|
H4rmony Dataset - https://huggingface.co/datasets/neovalle/H4rmony_dpo
|