初始化项目,由ModelHub XC社区提供模型
Model: integration1857/prescription-simplifier-mistral7b Source: Original Platform
This commit is contained in:
228
README.md
Normal file
228
README.md
Normal file
@@ -0,0 +1,228 @@
|
||||
---
|
||||
library_name: transformers
|
||||
tags: []
|
||||
---
|
||||
|
||||
# Model Card for Model ID
|
||||
|
||||
## Model Details
|
||||
|
||||
### Model Description
|
||||
|
||||
A fine-tuned version of Mistral-7B-Instruct-v0.3, trained to convert complex medical prescriptions into simple,
|
||||
patient-friendly explanations. Fine-tuned using QLoRA on Kaggle T4×2 GPUs with 8 hand-crafted prescription→explanation training
|
||||
examples covering Amoxicillin, Metformin, Lisinopril, Atorvastatin, Salbutamol, Sertraline, Warfarin, and Pantoprazole.
|
||||
|
||||
- **Developed by:** Madhukar Kumar
|
||||
- **Funded by [optional]:** Self Funded
|
||||
- **Shared by [optional]:** Madhukar Kumar
|
||||
- **Model type:** Causal Language Model (LLM) — fine-tuned using QLoRA (4-bit NF4 quantization + LoRA adapters) on mistralai/Mistral-7B-Instruct-v0.3
|
||||
- **Language(s) (NLP):** English
|
||||
- **License:** Apache 2.0 (inherited from Mistral-7B-Instruct-v0.3)
|
||||
- **Finetuned from model [optional]:** mistralai/Mistral-7B-Instruct-v0.3
|
||||
|
||||
### Model Sources [optional]
|
||||
|
||||
- **Repository:** (https://huggingface.co/integration1857/prescription-simplifier-mistral7b)
|
||||
- **Paper [optional]:** https://medium.com/p/49e94536f72b
|
||||
- **Demo [optional]:** (https://d6zhmy6z4ifp0.cloudfront.net)
|
||||
|
||||
## Uses
|
||||
|
||||
### Direct Use
|
||||
|
||||
This model is designed to convert complex medical prescription text into simple, plain-language explanations for patients.
|
||||
It can be used directly via the HuggingFace Inference API or integrated into healthcare applications, pharmacy portals,
|
||||
or patient-facing tools.
|
||||
|
||||
|
||||
### Downstream Use [optional]
|
||||
|
||||
Can be fine-tuned further on larger prescription datasets for improved accuracy.
|
||||
Can be integrated into hospital information systems, pharmacy apps, or telemedicine platforms to improve patient health literacy.
|
||||
|
||||
|
||||
|
||||
### Out-of-Scope Use
|
||||
|
||||
This model is NOT intended for clinical decision-making, medical diagnosis, or treatment recommendations.
|
||||
It should not replace professional medical advice. Not suitable for prescriptions in languages other than English.
|
||||
|
||||
|
||||
|
||||
## Bias, Risks, and Limitations
|
||||
|
||||
Trained on only 8 examples — limited coverage of drug types and medical conditions
|
||||
May produce inaccurate explanations for uncommon medications
|
||||
Does not account for patient-specific factors such as allergies or drug interactions
|
||||
English only — not suitable for multilingual use cases
|
||||
Should always be used alongside professional medical guidance
|
||||
|
||||
|
||||
### Recommendations
|
||||
|
||||
Always display a medical disclaimer alongside model outputs.
|
||||
Never use this model as a substitute for pharmacist or physician consultation.
|
||||
Outputs should be reviewed by a healthcare professional before deployment in clinical settings.
|
||||
|
||||
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
|
||||
More information needed for further recommendations.
|
||||
|
||||
## How to Get Started with the Model
|
||||
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
import torch
|
||||
|
||||
model_id = "integration1857/prescription-simplifier-mistral7b"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_id,
|
||||
torch_dtype=torch.float16,
|
||||
device_map="auto"
|
||||
)
|
||||
|
||||
prompt = """[INST] Convert this prescription into patient-friendly language:
|
||||
Amoxicillin 500mg TID x 7 days [/INST]"""
|
||||
|
||||
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
||||
outputs = model.generate(**inputs, max_new_tokens=300, temperature=0.3)
|
||||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||
|
||||
|
||||
|
||||
## Training Details
|
||||
|
||||
### Training Data
|
||||
|
||||
8 hand-crafted prescription→explanation pairs covering: Amoxicillin, Metformin, Lisinopril, Atorvastatin, Salbutamol, Sertraline, Warfarin, and Pantoprazole.
|
||||
Each example follows the Mistral [INST]...[/INST] instruction format.
|
||||
|
||||
|
||||
### Training Procedure
|
||||
|
||||
|
||||
#### Preprocessing [optional]
|
||||
|
||||
Prescriptions formatted using Mistral instruct template.
|
||||
Tokenized using the Mistral-7B-v0.3 tokenizer with a maximum sequence length of 512 tokens.
|
||||
|
||||
#### Training Hyperparameters
|
||||
|
||||
- **Training regime:**
|
||||
|
||||
bf16 mixed precision
|
||||
LoRA rank (r): 16
|
||||
LoRA alpha: 32
|
||||
LoRA dropout: 0.05
|
||||
Target modules: q_proj, v_proj, k_proj, o_proj
|
||||
Batch size: 2
|
||||
Gradient accumulation steps: 4
|
||||
Learning rate: 2e-4
|
||||
Epochs: 3
|
||||
Optimizer: paged_adamw_32bit
|
||||
|
||||
#### Speeds, Sizes, Times [optional]
|
||||
|
||||
Training time: 4.6 minutes
|
||||
Hardware: Kaggle T4×2 GPUs (16GB VRAM each)
|
||||
Trainable parameters: 41.94M (1.11% of 3.78B total)
|
||||
Base model size: ~14GB (fp16)
|
||||
|
||||
|
||||
## Evaluation
|
||||
|
||||
|
||||
### Testing Data, Factors & Metrics
|
||||
|
||||
#### Testing Data
|
||||
|
||||
Held-out prescription examples not seen during training, covering similar drug categories.
|
||||
|
||||
#### Factors
|
||||
|
||||
Evaluated on clarity of explanation, accuracy of dosage instructions, and completeness of warnings.
|
||||
|
||||
|
||||
#### Metrics
|
||||
|
||||
ROUGE-1 score used to measure overlap between generated explanations and reference explanations.
|
||||
|
||||
|
||||
### Results
|
||||
|
||||
ROUGE-1: 0.51
|
||||
Training loss: 1.78
|
||||
|
||||
#### Summary
|
||||
|
||||
The model achieves reasonable performance on the prescription simplification task given the very small training set.
|
||||
A larger, more diverse dataset would significantly improve generalisation.
|
||||
|
||||
|
||||
|
||||
## Model Examination [optional]
|
||||
|
||||
|
||||
## Environmental Impact
|
||||
|
||||
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
||||
|
||||
- **Hardware Type:** [NVIDIA Tesla T4 ×2]
|
||||
- **Hours used:** [0.08 hours (4.6 minutes)]
|
||||
- **Cloud Provider:** [Kaggle (Google Cloud backend)]
|
||||
- **Compute Region:** [US]
|
||||
- **Carbon Emitted:** [(< 0.01 kg CO₂ estimated)]
|
||||
|
||||
## Technical Specifications [optional]
|
||||
|
||||
### Model Architecture and Objective
|
||||
|
||||
Based on Mistral-7B-Instruct-v0.3 (decoder-only transformer).
|
||||
Fine-tuned with QLoRA — 4-bit NF4 quantization via bitsandbytes with LoRA adapter layers injected into attention modules.
|
||||
Objective: causal language modelling on prescription→explanation pairs.
|
||||
|
||||
### Compute Infrastructure
|
||||
|
||||
Kaggle Notebooks — free tier T4×2 GPU environment.
|
||||
|
||||
#### Hardware
|
||||
|
||||
2× NVIDIA Tesla T4 (16GB VRAM each)
|
||||
|
||||
#### Software
|
||||
|
||||
transformers, peft, trl, bitsandbytes, accelerate, torch 2.0
|
||||
|
||||
## Citation [optional]
|
||||
|
||||
**BibTeX:**
|
||||
|
||||
@misc{prescription-simplifier-2025,
|
||||
author = {integration1857},
|
||||
title = {Prescription Simplifier — Mistral-7B Fine-tuned for Patient-Friendly Medical Explanations},
|
||||
year = {2025},
|
||||
publisher = {HuggingFace},
|
||||
url = {https://huggingface.co/integration1857/prescription-simplifier-mistral7b}
|
||||
}
|
||||
|
||||
**APA:**
|
||||
|
||||
integration1857. (2025).Prescription Simplifier — Mistral-7B Fine-tuned for Patient-Friendly Medical Explanations.
|
||||
HuggingFace. https://huggingface.co/integration1857/prescription-simplifier-mistral7b
|
||||
|
||||
## Glossary [optional]
|
||||
|
||||
https://medium.com/p/49e94536f72b
|
||||
|
||||
## More Information [optional]
|
||||
|
||||
(https://www.linkedin.com/pulse/doctors-write-code-i-built-ai-translate-madhukar-kumar-3f9cc/?trackingId=v%2BIJoWuLQPuJPt0yNtxe6Q%3D%3D)
|
||||
|
||||
## Model Card Authors [optional]
|
||||
|
||||
Madhukar Kumar
|
||||
|
||||
## Model Card Contact
|
||||
|
||||
Via HuggingFace profile: https://huggingface.co/integration1857
|
||||
Reference in New Issue
Block a user