97 lines
3.3 KiB
Markdown
97 lines
3.3 KiB
Markdown
|
---
|
|||
|
|
library_name: transformers
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
license: mit
|
|||
|
|
base_model:
|
|||
|
|
- Qwen/Qwen2-7B-Instruct
|
|||
|
|
tags:
|
|||
|
|
- medical
|
|||
|
|
datasets:
|
|||
|
|
- WaltonFuture/Diabetica-SFT
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Diabetica-7B
|
|||
|
|
|
|||
|
|
<div align="center">
|
|||
|
|
<h2>
|
|||
|
|
Diabetica: Adapting Large Language Model to Enhance Multiple Medical Tasks in Diabetes Care and Management
|
|||
|
|
</h2>
|
|||
|
|
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<p align="center">
|
|||
|
|
<a href="https://github.com/waltonfuture/Diabetica" target="_blank">Code</a>｜<a href="https://arxiv.org/pdf/2409.13191" target="_blank">Paper</a> | <a href="https://huggingface.co/datasets/WaltonFuture/Diabetica-SFT" target="_blank">Dataset</a><br>
|
|||
|
|
</p>
|
|||
|
|
|
|||
|
|
## Introduction
|
|||
|
|
|
|||
|
|
Hello! Welcome to the huggingface repository for [Diabetica](https://arxiv.org/pdf/2409.13191).
|
|||
|
|
|
|||
|
|
Our study introduced a reproducible framework for developing a specialized LLM capable of handling various diabetes tasks. We present three key contributions:
|
|||
|
|
|
|||
|
|
- High-performance domain-specific model: Compared with previous generic LLMs, our model Diabetica, showed superior performance across a broad range of diabetes-related tasks, including diagnosis, treatment recommendations, medication management, lifestyle advice, patient education, and so on.
|
|||
|
|
|
|||
|
|
- Reproducible framework: We offered a detailed method for creating specialized medical LLMs using open-source models, curated disease-specific datasets, and fine-tuning techniques. This approach can be adapted to other medical fields, potentially accelerating AI-assisted care development.
|
|||
|
|
|
|||
|
|
- Comprehensive evaluation: We designed comprehensive benchmarks and conducted clinical trials to validate the model's effectiveness in clinical applications. This ensured our model's practical utility and sets a new standard for evaluating AI tools in diabetes care.
|
|||
|
|
|
|||
|
|
Please refer to our [GitHub Repo](https://github.com/waltonfuture/Diabetica) for more details.
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Model Inference
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|||
|
|
import torch
|
|||
|
|
|
|||
|
|
device = "cuda" # the device to load the model onto
|
|||
|
|
model_path = 'WaltonFuture/Diabetica-7B'
|
|||
|
|
|
|||
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|||
|
|
model_path,
|
|||
|
|
torch_dtype="auto",
|
|||
|
|
device_map="auto"
|
|||
|
|
)
|
|||
|
|
tokenizer = AutoTokenizer.from_pretrained(model_path)
|
|||
|
|
|
|||
|
|
def model_output(content):
|
|||
|
|
messages = [
|
|||
|
|
{"role": "system", "content": "You are a helpful assistant."},
|
|||
|
|
{"role": "user", "content": content}
|
|||
|
|
]
|
|||
|
|
text = tokenizer.apply_chat_template(
|
|||
|
|
messages,
|
|||
|
|
tokenize=False,
|
|||
|
|
add_generation_prompt=True
|
|||
|
|
)
|
|||
|
|
model_inputs = tokenizer([text], return_tensors="pt").to(device)
|
|||
|
|
generated_ids = model.generate(
|
|||
|
|
model_inputs.input_ids,
|
|||
|
|
max_new_tokens=2048,
|
|||
|
|
do_sample=True,
|
|||
|
|
)
|
|||
|
|
generated_ids = [
|
|||
|
|
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
|||
|
|
]
|
|||
|
|
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
|||
|
|
return response
|
|||
|
|
|
|||
|
|
prompt = "Hello! Please tell me something about diabetes."
|
|||
|
|
|
|||
|
|
response = model_output(prompt)
|
|||
|
|
print(response)
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Citation
|
|||
|
|
```
|
|||
|
|
@article{wei2024adapted,
|
|||
|
|
title={An adapted large language model facilitates multiple medical tasks in diabetes care},
|
|||
|
|
author={Wei, Lai and Ying, Zhen and He, Muyang and Chen, Yutong and Yang, Qian and Hong, Yanzhe and Lu, Jiaping and Li, Xiaoying and Huang, Weiran and Chen, Ying},
|
|||
|
|
journal={arXiv preprint arXiv:2409.13191},
|
|||
|
|
year={2024}
|
|||
|
|
}
|
|||
|
|
```
|