HPAI-BSC/Llama3-Aloe-8B-Alpha
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Ashwin Kumar
2024-05-14 09:00:09 +00:00
committed by system
parent 8353550720
commit da62bcec82
1 changed files with 11 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
README.md
View File

@@ -22,12 +22,13 @@ library_name: transformers
tags: tags:
- biology - biology
- medical - medical
pipeline_tag: question-answering
--- ---
# Aloe: A New Family of Healthcare LLMs # Aloe: A New Family of Healthcare LLMs
Aloe is a new family of healthcare LLMs that is highly competitive with all previous open models of its range and reaches state-of-the-art results at its size by using model merging and advanced prompting strategies. Aloe scores high in metrics measuring ethics and factuality, thanks to a combined red teaming and alignment effort. Complete training details, model merging configurations, and all training data (including synthetically generated data) will be shared. Additionally, the prompting repository used in this work to produce state-of-the-art results during inference will also be shared. Aloe comes with a healthcare-specific risk assessment to contribute to the safe use and deployment of such systems. Aloe is a new family of healthcare LLMs that is highly competitive with all previous open models of its range and reaches state-of-the-art results at its size by using model merging and advanced prompting strategies. Aloe scores high in metrics measuring ethics and factuality, thanks to a combined red teaming and alignment effort. Complete training details, model merging configurations, and all training data (including synthetically generated data) will be shared. Additionally, the prompting repository used in this work to produce state-of-the-art results during inference will also be shared. Aloe comes with a healthcare-specific risk assessment to contribute to the safe use and deployment of such systems.
<img src="https://cdn-uploads.huggingface.co/production/uploads/62f7a16192950415b637e201/HMD6WEoqqrAV8Ng_fAcnN.png" width="95%"> <img src="https://cdn-uploads.huggingface.co/production/uploads/62972c4979f193515da1d38e/xlssx5_3_kLQlJlmE-aya.png" width="95%">
## Model Details ## Model Details
@@ -168,6 +169,9 @@ Supervised fine-tuning on top of Llama 3 8B using medical and general domain dat
### Training Data ### Training Data
- Medical domain datasets, including synthetic data generated using Mixtral-8x7B and Genstruct - Medical domain datasets, including synthetic data generated using Mixtral-8x7B and Genstruct
- HPAI-BSC/pubmedqa-cot
- HPAI-BSC/medqa-cot
- HPAI-BSC/medmcqa-cot
- LDJnr/Capybara - LDJnr/Capybara
- hkust-nlp/deita-10k-v0 - hkust-nlp/deita-10k-v0
- jondurbin/airoboros-3.2 - jondurbin/airoboros-3.2
@@ -212,6 +216,9 @@ With the help of prompting techniques the performance of Llama3-Aloe-8B-Alpha is
- **Compute Region:** Spain - **Compute Region:** Spain
- **Carbon Emitted:** 439.25kg - **Carbon Emitted:** 439.25kg
## Model Card Authors
[Ashwin Kumar Gururajan](https://huggingface.co/G-AshwinKumar)
## Model Card Contact ## Model Card Contact
mailto:hpai@bsc.es mailto:hpai@bsc.es
@@ -220,6 +227,7 @@ mailto:hpai@bsc.es
If you use this repository in a published work, please cite the following papers as source: If you use this repository in a published work, please cite the following papers as source:
```
@misc{gururajan2024aloe, @misc{gururajan2024aloe,
title={Aloe: A Family of Fine-tuned Open Healthcare LLMs}, title={Aloe: A Family of Fine-tuned Open Healthcare LLMs},
author={Ashwin Kumar Gururajan and Enrique Lopez-Cuena and Jordi Bayarri-Planas and Adrian Tormos and Daniel Hinjos and Pablo Bernabeu-Perez and Anna Arias-Duart and Pablo Agustin Martin-Torres and Lucia Urcelay-Ganzabal and Marta Gonzalez-Mallo and Sergio Alvarez-Napagao and Eduard Ayguadé-Parra and Ulises Cortés Dario Garcia-Gasulla}, author={Ashwin Kumar Gururajan and Enrique Lopez-Cuena and Jordi Bayarri-Planas and Adrian Tormos and Daniel Hinjos and Pablo Bernabeu-Perez and Anna Arias-Duart and Pablo Agustin Martin-Torres and Lucia Urcelay-Ganzabal and Marta Gonzalez-Mallo and Sergio Alvarez-Napagao and Eduard Ayguadé-Parra and Ulises Cortés Dario Garcia-Gasulla},
@@ -227,4 +235,5 @@ If you use this repository in a published work, please cite the following papers
eprint={2405.01886}, eprint={2405.01886},
archivePrefix={arXiv}, archivePrefix={arXiv},
primaryClass={cs.CL} primaryClass={cs.CL}
} }
```