47 lines
1.8 KiB
Markdown
47 lines
1.8 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
base_model:
|
||
|
|
- mistralai/Mistral-7B-v0.1
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- clinical trial
|
||
|
|
- foundation model
|
||
|
|
---
|
||
|
|
|
||
|
|
# Model Card for Panacea-7B-Chat
|
||
|
|
The Panacea-7B-Chat is a foundation model for clinical trial search, summarization, design, and recruitment. It was equipped with clinical knowledge by being trained on 793,279 clinical trial design documents
|
||
|
|
worldwide and 1,113,207 clinical study papers. It shows superior performances than various open-sourced LLMs and medical LLMs on clinical trial tasks.
|
||
|
|
|
||
|
|
For full details of this model please read our [paper](https://arxiv.org/abs/2407.11007).
|
||
|
|
|
||
|
|
## Model Training
|
||
|
|
Panacea is trained from [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1). The training of Panacea consists of an alignment step and an instruction-tuning step.
|
||
|
|
* Alignment step: continued pre-training on a large collection of trial documents and trial-related scientific papers. This step adapts Panacea to the vocabulary commonly used in clinical trials.
|
||
|
|
* Instruction-tuning step: further enables Panacea to comprehend the user explanation of the task definition and the output requirement.
|
||
|
|
|
||
|
|
Load the model in the following way (same as Mistral):
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
import torch
|
||
|
|
|
||
|
|
model_id = 'linjc16/Panacea-7B-Chat'
|
||
|
|
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
|
|
```
|
||
|
|
|
||
|
|
## Citation
|
||
|
|
If you find our paper or models helpful, please consider cite as follows:
|
||
|
|
|
||
|
|
```bibtex
|
||
|
|
@article{lin2024panacea,
|
||
|
|
title={Panacea: A foundation model for clinical trial search, summarization, design, and recruitment},
|
||
|
|
author={Lin, Jiacheng and Xu, Hanwen and Wang, Zifeng and Wang, Sheng and Sun, Jimeng},
|
||
|
|
journal={arXiv preprint arXiv:2407.11007},
|
||
|
|
year={2024}
|
||
|
|
}
|
||
|
|
```
|