228 lines
6.8 KiB
Markdown
228 lines
6.8 KiB
Markdown
---
|
||
library_name: transformers
|
||
tags: []
|
||
---
|
||
|
||
# Model Card for Model ID
|
||
|
||
## Model Details
|
||
|
||
### Model Description
|
||
|
||
A fine-tuned version of Mistral-7B-Instruct-v0.3, trained to convert complex medical prescriptions into simple,
|
||
patient-friendly explanations. Fine-tuned using QLoRA on Kaggle T4×2 GPUs with 8 hand-crafted prescription→explanation training
|
||
examples covering Amoxicillin, Metformin, Lisinopril, Atorvastatin, Salbutamol, Sertraline, Warfarin, and Pantoprazole.
|
||
|
||
- **Developed by:** Madhukar Kumar
|
||
- **Funded by [optional]:** Self Funded
|
||
- **Shared by [optional]:** Madhukar Kumar
|
||
- **Model type:** Causal Language Model (LLM) — fine-tuned using QLoRA (4-bit NF4 quantization + LoRA adapters) on mistralai/Mistral-7B-Instruct-v0.3
|
||
- **Language(s) (NLP):** English
|
||
- **License:** Apache 2.0 (inherited from Mistral-7B-Instruct-v0.3)
|
||
- **Finetuned from model [optional]:** mistralai/Mistral-7B-Instruct-v0.3
|
||
|
||
### Model Sources [optional]
|
||
|
||
- **Repository:** (https://huggingface.co/integration1857/prescription-simplifier-mistral7b)
|
||
- **Paper [optional]:** https://medium.com/p/49e94536f72b
|
||
- **Demo [optional]:** (https://d6zhmy6z4ifp0.cloudfront.net)
|
||
|
||
## Uses
|
||
|
||
### Direct Use
|
||
|
||
This model is designed to convert complex medical prescription text into simple, plain-language explanations for patients.
|
||
It can be used directly via the HuggingFace Inference API or integrated into healthcare applications, pharmacy portals,
|
||
or patient-facing tools.
|
||
|
||
|
||
### Downstream Use [optional]
|
||
|
||
Can be fine-tuned further on larger prescription datasets for improved accuracy.
|
||
Can be integrated into hospital information systems, pharmacy apps, or telemedicine platforms to improve patient health literacy.
|
||
|
||
|
||
|
||
### Out-of-Scope Use
|
||
|
||
This model is NOT intended for clinical decision-making, medical diagnosis, or treatment recommendations.
|
||
It should not replace professional medical advice. Not suitable for prescriptions in languages other than English.
|
||
|
||
|
||
|
||
## Bias, Risks, and Limitations
|
||
|
||
Trained on only 8 examples — limited coverage of drug types and medical conditions
|
||
May produce inaccurate explanations for uncommon medications
|
||
Does not account for patient-specific factors such as allergies or drug interactions
|
||
English only — not suitable for multilingual use cases
|
||
Should always be used alongside professional medical guidance
|
||
|
||
|
||
### Recommendations
|
||
|
||
Always display a medical disclaimer alongside model outputs.
|
||
Never use this model as a substitute for pharmacist or physician consultation.
|
||
Outputs should be reviewed by a healthcare professional before deployment in clinical settings.
|
||
|
||
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
|
||
More information needed for further recommendations.
|
||
|
||
## How to Get Started with the Model
|
||
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
import torch
|
||
|
||
model_id = "integration1857/prescription-simplifier-mistral7b"
|
||
|
||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_id,
|
||
torch_dtype=torch.float16,
|
||
device_map="auto"
|
||
)
|
||
|
||
prompt = """[INST] Convert this prescription into patient-friendly language:
|
||
Amoxicillin 500mg TID x 7 days [/INST]"""
|
||
|
||
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
||
outputs = model.generate(**inputs, max_new_tokens=300, temperature=0.3)
|
||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
|
||
|
||
|
||
## Training Details
|
||
|
||
### Training Data
|
||
|
||
8 hand-crafted prescription→explanation pairs covering: Amoxicillin, Metformin, Lisinopril, Atorvastatin, Salbutamol, Sertraline, Warfarin, and Pantoprazole.
|
||
Each example follows the Mistral [INST]...[/INST] instruction format.
|
||
|
||
|
||
### Training Procedure
|
||
|
||
|
||
#### Preprocessing [optional]
|
||
|
||
Prescriptions formatted using Mistral instruct template.
|
||
Tokenized using the Mistral-7B-v0.3 tokenizer with a maximum sequence length of 512 tokens.
|
||
|
||
#### Training Hyperparameters
|
||
|
||
- **Training regime:**
|
||
|
||
bf16 mixed precision
|
||
LoRA rank (r): 16
|
||
LoRA alpha: 32
|
||
LoRA dropout: 0.05
|
||
Target modules: q_proj, v_proj, k_proj, o_proj
|
||
Batch size: 2
|
||
Gradient accumulation steps: 4
|
||
Learning rate: 2e-4
|
||
Epochs: 3
|
||
Optimizer: paged_adamw_32bit
|
||
|
||
#### Speeds, Sizes, Times [optional]
|
||
|
||
Training time: 4.6 minutes
|
||
Hardware: Kaggle T4×2 GPUs (16GB VRAM each)
|
||
Trainable parameters: 41.94M (1.11% of 3.78B total)
|
||
Base model size: ~14GB (fp16)
|
||
|
||
|
||
## Evaluation
|
||
|
||
|
||
### Testing Data, Factors & Metrics
|
||
|
||
#### Testing Data
|
||
|
||
Held-out prescription examples not seen during training, covering similar drug categories.
|
||
|
||
#### Factors
|
||
|
||
Evaluated on clarity of explanation, accuracy of dosage instructions, and completeness of warnings.
|
||
|
||
|
||
#### Metrics
|
||
|
||
ROUGE-1 score used to measure overlap between generated explanations and reference explanations.
|
||
|
||
|
||
### Results
|
||
|
||
ROUGE-1: 0.51
|
||
Training loss: 1.78
|
||
|
||
#### Summary
|
||
|
||
The model achieves reasonable performance on the prescription simplification task given the very small training set.
|
||
A larger, more diverse dataset would significantly improve generalisation.
|
||
|
||
|
||
|
||
## Model Examination [optional]
|
||
|
||
|
||
## Environmental Impact
|
||
|
||
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
||
|
||
- **Hardware Type:** [NVIDIA Tesla T4 ×2]
|
||
- **Hours used:** [0.08 hours (4.6 minutes)]
|
||
- **Cloud Provider:** [Kaggle (Google Cloud backend)]
|
||
- **Compute Region:** [US]
|
||
- **Carbon Emitted:** [(< 0.01 kg CO₂ estimated)]
|
||
|
||
## Technical Specifications [optional]
|
||
|
||
### Model Architecture and Objective
|
||
|
||
Based on Mistral-7B-Instruct-v0.3 (decoder-only transformer).
|
||
Fine-tuned with QLoRA — 4-bit NF4 quantization via bitsandbytes with LoRA adapter layers injected into attention modules.
|
||
Objective: causal language modelling on prescription→explanation pairs.
|
||
|
||
### Compute Infrastructure
|
||
|
||
Kaggle Notebooks — free tier T4×2 GPU environment.
|
||
|
||
#### Hardware
|
||
|
||
2× NVIDIA Tesla T4 (16GB VRAM each)
|
||
|
||
#### Software
|
||
|
||
transformers, peft, trl, bitsandbytes, accelerate, torch 2.0
|
||
|
||
## Citation [optional]
|
||
|
||
**BibTeX:**
|
||
|
||
@misc{prescription-simplifier-2025,
|
||
author = {integration1857},
|
||
title = {Prescription Simplifier — Mistral-7B Fine-tuned for Patient-Friendly Medical Explanations},
|
||
year = {2025},
|
||
publisher = {HuggingFace},
|
||
url = {https://huggingface.co/integration1857/prescription-simplifier-mistral7b}
|
||
}
|
||
|
||
**APA:**
|
||
|
||
integration1857. (2025).Prescription Simplifier — Mistral-7B Fine-tuned for Patient-Friendly Medical Explanations.
|
||
HuggingFace. https://huggingface.co/integration1857/prescription-simplifier-mistral7b
|
||
|
||
## Glossary [optional]
|
||
|
||
https://medium.com/p/49e94536f72b
|
||
|
||
## More Information [optional]
|
||
|
||
(https://www.linkedin.com/pulse/doctors-write-code-i-built-ai-translate-madhukar-kumar-3f9cc/?trackingId=v%2BIJoWuLQPuJPt0yNtxe6Q%3D%3D)
|
||
|
||
## Model Card Authors [optional]
|
||
|
||
Madhukar Kumar
|
||
|
||
## Model Card Contact
|
||
|
||
Via HuggingFace profile: https://huggingface.co/integration1857 |