115 lines
3.5 KiB
Markdown
115 lines
3.5 KiB
Markdown
|
---
|
|||
|
|
base_model: unsloth/qwen2.5-1.5b-unsloth-bnb-4bit
|
|||
|
|
tags:
|
|||
|
|
- text-generation-inference
|
|||
|
|
- transformers
|
|||
|
|
- unsloth
|
|||
|
|
- qwen2
|
|||
|
|
license: apache-2.0
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
datasets:
|
|||
|
|
- miriad/miriad-4.4M
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
# 🧠 MedScholar-1.5B
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
<img src="https://huggingface.co/yasserrmd/MedScholar-1.5B/resolve/main/banner.png" width="800"/>
|
|||
|
|
|
|||
|
|
**MedScholar-1.5B** is a compact, instruction-aligned medical question-answering model fine-tuned on 1 million randomly selected examples from the [MIRIAD-4.4M dataset](https://huggingface.co/datasets/miriad/miriad-4.4M). It is based on the [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) model and designed for efficient, in-context clinical knowledge exploration — **not diagnosis**.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 📌 Model Details
|
|||
|
|
|
|||
|
|
- **Base Model**: [Qwen2.5-1.5B-Instruct-unsloth-bnb-4bit](https://huggingface.co/unsloth/Qwen2.5-1.5B-Instruct-unsloth-bnb-4bit)
|
|||
|
|
- **Fine-tuning Dataset**: [MIRIAD-4.4M](https://huggingface.co/datasets/miriad/miriad-4.4M)
|
|||
|
|
- **Samples Used**: 1,000,000 examples randomly selected from the full set
|
|||
|
|
- **Prompt Style**: Minimal QA format (see below)
|
|||
|
|
- **Training Framework**: [Unsloth](https://github.com/unslothai/unsloth) with QLoRA
|
|||
|
|
- **License**: Apache-2.0 (inherits from base model); dataset is ODC-By 1.0
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 📋 Prompt Format
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
### Question:
|
|||
|
|
What is the role of LDL in cardiovascular health?
|
|||
|
|
|
|||
|
|
### Answer:
|
|||
|
|
LDL plays a central role in the development of atherosclerosis by delivering cholesterol to peripheral tissues...
|
|||
|
|
````
|
|||
|
|
|
|||
|
|
* The model expects the prompt to **end with `### Answer:`**, and will generate only the answer text.
|
|||
|
|
* Do **not include the answer in the prompt** during inference.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🔒 Dataset Consent & License
|
|||
|
|
|
|||
|
|
This model was fine-tuned using **randomly selected 1 million examples** from the [MIRIAD-4.4M dataset](https://huggingface.co/datasets/miriad/miriad-4.4M), which is released under the [ODC-By 1.0 License](https://opendatacommons.org/licenses/by/1-0/).
|
|||
|
|
|
|||
|
|
> **The MIRIAD dataset is intended exclusively for academic research and educational exploration.**
|
|||
|
|
> As stated by its authors:
|
|||
|
|
>
|
|||
|
|
> *“The outputs generated by models trained or fine-tuned on this dataset must not be used for medical diagnosis or decision-making involving real individuals.”*
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## ⚠️ Intended Use
|
|||
|
|
|
|||
|
|
**This model is for research, educational, and exploration purposes only. It is not a medical device and must not be used to provide clinical advice, diagnosis, or treatment.**
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 💡 Example Inference (Python)
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from transformers import pipeline
|
|||
|
|
|
|||
|
|
pipe = pipeline("text-generation", model="yasserrmd/MedScholar-1.5B", device=0)
|
|||
|
|
|
|||
|
|
prompt = """### Question:
|
|||
|
|
What are the symptoms of acute pancreatitis?
|
|||
|
|
|
|||
|
|
### Answer:
|
|||
|
|
"""
|
|||
|
|
|
|||
|
|
response = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7)
|
|||
|
|
print(response[0]["generated_text"])
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 🤝 Acknowledgements
|
|||
|
|
|
|||
|
|
* MIRIAD Dataset by Zheng et al. (2025) – [https://huggingface.co/datasets/miriad/miriad-4.4M](https://huggingface.co/datasets/miriad/miriad-4.4M)
|
|||
|
|
* Qwen2.5 by Alibaba – [https://huggingface.co/Qwen](https://huggingface.co/Qwen)
|
|||
|
|
* Training infrastructure: [Unsloth](https://github.com/unslothai/unsloth)
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## 📄 Citation
|
|||
|
|
|
|||
|
|
```bibtex
|
|||
|
|
@misc{yasser2025medscholar,
|
|||
|
|
title = {MedScholar-1.5B: Compact medical QA model fine-tuned on MIRIAD},
|
|||
|
|
author = {Mohamed Yasser},
|
|||
|
|
year = {2025},
|
|||
|
|
howpublished = {\url{https://huggingface.co/yasserrmd/MedScholar-1.5B}},
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|