adityak74/medfit-llm-3B
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Aditya Karnam
2025-09-15 17:20:23 +00:00
committed by system
parent 75d2a9c52e
commit fb4cb90509
1 changed files with 208 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

209
README.md
View File

@@ -4,4 +4,211 @@ language:
- en
base_model:
- meta-llama/Llama-3.2-3B-Instruct
---
library_name: transformers
tags:
- healthcare
- medical
- fine-tuned
- chatbot
- medical-qa
- domain-specific
- lora
- healthcare-ai
pipeline_tag: text-generation
datasets:
- synthetic-medical-qa
---
# MEDFIT-LLM-3B: Fine-tuned Llama-3.2-3B for Medical QA
MEDFIT-LLM-3B is a specialized language model fine-tuned from Meta's Llama-3.2-3B-Instruct for healthcare and medical question-answering applications. This model demonstrates significant improvements in direct answer capabilities and medical domain understanding through domain-focused fine-tuning.
## Model Details
### Model Description
MEDFIT-LLM-3B is a 3 billion parameter language model specifically optimized for healthcare chatbot applications. The model was fine-tuned using LoRA (Low-Rank Adaptation) techniques on a carefully curated dataset of healthcare-related questions and answers, resulting in enhanced performance for medical information dissemination and patient education.
- **Developed by:** Aditya Karnam Gururaj Rao, Arjun Jaggi, Sonam Naidu
- **Model type:** Causal Language Model (Fine-tuned)
- **Language(s):** English
- **License:** Llama 3.2 Community License
- **Finetuned from model:** meta-llama/Llama-3.2-3B-Instruct
- **Fine-tuning method:** LoRA (Low-Rank Adaptation)
- **Training framework:** MLX
### Model Sources
- **Repository:** https://huggingface.co/adityak74/medfit-llm-3B
- **Paper:** [MEDFIT-LLM: Medical Enhancements through Domain-Focused Fine Tuning of Small Language Models](https://github.com/adityak74/medfit-llm)
- **Code:** https://github.com/adityak74/medfit-llm
## Performance Highlights
Based on comprehensive evaluation against the base Llama-3.2-3B-Instruct model:
- **Direct Answer Improvement:** 30 percentage point increase (from 6.0% to 36.0%)
- **Response Structure:** 18% increase in numbered list usage for better organization
- **Overall Improvement Score:** 108.2 (highest among evaluated models)
- **Response Length:** Slight increase (+2.84%) with more comprehensive answers
## Uses
### Direct Use
MEDFIT-LLM-3B is designed for healthcare chatbot applications where accurate, well-structured medical information delivery is crucial. The model excels at:
- **Medical Question Answering:** Providing direct, accurate responses to healthcare queries
- **Patient Education:** Delivering structured, easy-to-understand medical information
- **Healthcare Information Dissemination:** Supporting healthcare providers with reliable AI assistance
- **Medical Chatbot Applications:** Serving as the backbone for healthcare conversational agents
### Downstream Use
The model can be integrated into:
- Healthcare mobile applications
- Medical information systems
- Patient support platforms
- Telemedicine chatbots
- Medical education tools
### Out-of-Scope Use
**Important:** This model is NOT intended for:
- **Medical diagnosis or treatment recommendations**
- **Emergency medical situations**
- **Replacement of professional medical advice**
- **Clinical decision-making without human oversight**
- **Prescription or medication recommendations**
## Training Details
### Training Data
The model was trained on a carefully curated dataset comprising:
- **Total samples:** 6,444 unique healthcare-related question-answer pairs
- **Training set:** 5,155 samples
- **Validation set:** 644 samples
- **Test set:** 645 samples
The dataset was created using:
- **Synthetic data generation:** 10,000 initial samples generated using Phi-4
- **Domain-specific curation:** Healthcare-focused questions derived from existing research
- **Deduplication:** Filtered to remove duplicates, resulting in 6,444 unique samples
### Training Procedure
#### Fine-tuning Method
- **Technique:** LoRA (Low-Rank Adaptation)
- **Framework:** MLX
- **Base model:** meta-llama/Llama-3.2-3B-Instruct
- **Focus:** Healthcare domain specialization
#### Training Hyperparameters
- **Fine-tuning approach:** Domain-focused LoRA adaptation
- **Dataset split:** 80% training, 10% validation, 10% testing
- **Training regime:** Optimized for healthcare question-answering performance
## Evaluation
### Testing Data & Metrics
The model was evaluated on:
- **50 healthcare-specific validation questions**
- **Comparative analysis** against base Llama-3.2-3B-Instruct
- **Multi-dimensional assessment** including direct answer capability, response structure, and generation efficiency
### Key Results
**Direct Answer Performance:**
- Base model: 6.0% direct answer rate
- Fine-tuned model: 36.0% direct answer rate
- **Improvement: +30.0 percentage points**
**Response Quality:**
- Enhanced structure with increased use of numbered lists (+18%)
- Improved organization and systematic presentation
- Better alignment with healthcare communication standards
**Generation Efficiency:**
- Slight increase in generation time (+1.6%)
- Trade-off between response quality and speed
- Overall positive impact on response comprehensiveness
## Bias, Risks, and Limitations
### Limitations
- **Not a substitute for professional medical advice**
- **May generate plausible-sounding but incorrect medical information**
- **Limited to English language medical contexts**
- **Training data may not cover all medical specialties equally**
- **Performance may vary across different healthcare subdomains**
### Recommendations
- **Always verify medical information** with qualified healthcare professionals
- **Use as a supplementary tool** rather than primary medical resource
- **Implement human oversight** in all healthcare applications
- **Regular updates** needed to maintain medical accuracy as knowledge evolves
- **Consider integration with retrieval-augmented generation (RAG)** for enhanced factual accuracy
## How to Get Started with the Model
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
# Load the model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("adityak74/medfit-llm-3B")
model = AutoModelForCausalLM.from_pretrained("adityak74/medfit-llm-3B")
# Example usage
prompt = "What are the common symptoms of diabetes?"
inputs = tokenizer.encode(prompt, return_tensors="pt")
outputs = model.generate(inputs, max_length=200, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```
## Environmental Impact
The fine-tuning process utilized efficient LoRA techniques to minimize computational requirements while maximizing performance improvements. This approach reduces the environmental impact compared to full model training while achieving significant domain-specific enhancements.
## Citation
**BibTeX:**
```bibtex
@inproceedings{rao2025medfit,
title={MEDFIT-LLM: Medical Enhancements through Domain-Focused Fine Tuning of Small Language Models},
author={Rao, Aditya Karnam Gururaj and Jaggi, Arjun and Naidu, Sonam},
booktitle={2025 2nd International Conference on Research Methodologies in Knowledge Management, Artificial Intelligence and Telecommunication Engineering (RMKMATE)},
year={2025},
organization={IEEE}
}
```
**APA:**
Rao, A. K. G., Jaggi, A., & Naidu, S. (2025). MEDFIT-LLM: Medical Enhancements through Domain-Focused Fine Tuning of Small Language Models. In *2025 2nd International Conference on Research Methodologies in Knowledge Management, Artificial Intelligence and Telecommunication Engineering (RMKMATE)*. IEEE.
## Glossary
- **QA:** Question Answering
- **EHR:** Electronic Health Record
- **LoRA:** Low-Rank Adaptation - an efficient fine-tuning technique
- **MLX:** Machine Learning framework used for training
- **Direct Answer Rate:** Percentage of responses that begin with direct answers rather than preambles
## Model Card Authors
- **Aditya Karnam Gururaj Rao** (Zefr Inc, LA, USA)
- **Arjun Jaggi** (HCLTech, LA, USA)
- **Sonam Naidu** (LexisNexis, USA)
## Model Card Contact
- **Primary Contact:** https://huggingface.co/adityak74
- **Email:** akarnam37@gmail.com
- **GitHub:** https://github.com/adityak74/medfit-llm
## Disclaimer
This model is designed for educational and informational purposes in healthcare contexts. It should not be used as a substitute for professional medical advice, diagnosis, or treatment. Always seek the advice of qualified healthcare providers with questions regarding medical conditions or treatments.