From fb4cb905099e414496edc7d8a9c75d659a096196 Mon Sep 17 00:00:00 2001 From: Aditya Karnam Date: Mon, 15 Sep 2025 17:20:23 +0000 Subject: [PATCH] Update README.md --- README.md | 209 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 208 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index e118631..8b63e94 100644 --- a/README.md +++ b/README.md @@ -4,4 +4,211 @@ language: - en base_model: - meta-llama/Llama-3.2-3B-Instruct ---- \ No newline at end of file +library_name: transformers +tags: +- healthcare +- medical +- fine-tuned +- chatbot +- medical-qa +- domain-specific +- lora +- healthcare-ai +pipeline_tag: text-generation +datasets: +- synthetic-medical-qa +--- + +# MEDFIT-LLM-3B: Fine-tuned Llama-3.2-3B for Medical QA + +MEDFIT-LLM-3B is a specialized language model fine-tuned from Meta's Llama-3.2-3B-Instruct for healthcare and medical question-answering applications. This model demonstrates significant improvements in direct answer capabilities and medical domain understanding through domain-focused fine-tuning. + +## Model Details + +### Model Description + +MEDFIT-LLM-3B is a 3 billion parameter language model specifically optimized for healthcare chatbot applications. The model was fine-tuned using LoRA (Low-Rank Adaptation) techniques on a carefully curated dataset of healthcare-related questions and answers, resulting in enhanced performance for medical information dissemination and patient education. + +- **Developed by:** Aditya Karnam Gururaj Rao, Arjun Jaggi, Sonam Naidu +- **Model type:** Causal Language Model (Fine-tuned) +- **Language(s):** English +- **License:** Llama 3.2 Community License +- **Finetuned from model:** meta-llama/Llama-3.2-3B-Instruct +- **Fine-tuning method:** LoRA (Low-Rank Adaptation) +- **Training framework:** MLX + +### Model Sources + +- **Repository:** https://huggingface.co/adityak74/medfit-llm-3B +- **Paper:** [MEDFIT-LLM: Medical Enhancements through Domain-Focused Fine Tuning of Small Language Models](https://github.com/adityak74/medfit-llm) +- **Code:** https://github.com/adityak74/medfit-llm + +## Performance Highlights + +Based on comprehensive evaluation against the base Llama-3.2-3B-Instruct model: + +- **Direct Answer Improvement:** 30 percentage point increase (from 6.0% to 36.0%) +- **Response Structure:** 18% increase in numbered list usage for better organization +- **Overall Improvement Score:** 108.2 (highest among evaluated models) +- **Response Length:** Slight increase (+2.84%) with more comprehensive answers + +## Uses + +### Direct Use + +MEDFIT-LLM-3B is designed for healthcare chatbot applications where accurate, well-structured medical information delivery is crucial. The model excels at: + +- **Medical Question Answering:** Providing direct, accurate responses to healthcare queries +- **Patient Education:** Delivering structured, easy-to-understand medical information +- **Healthcare Information Dissemination:** Supporting healthcare providers with reliable AI assistance +- **Medical Chatbot Applications:** Serving as the backbone for healthcare conversational agents + +### Downstream Use + +The model can be integrated into: +- Healthcare mobile applications +- Medical information systems +- Patient support platforms +- Telemedicine chatbots +- Medical education tools + +### Out-of-Scope Use + +**Important:** This model is NOT intended for: +- **Medical diagnosis or treatment recommendations** +- **Emergency medical situations** +- **Replacement of professional medical advice** +- **Clinical decision-making without human oversight** +- **Prescription or medication recommendations** + +## Training Details + +### Training Data + +The model was trained on a carefully curated dataset comprising: +- **Total samples:** 6,444 unique healthcare-related question-answer pairs +- **Training set:** 5,155 samples +- **Validation set:** 644 samples +- **Test set:** 645 samples + +The dataset was created using: +- **Synthetic data generation:** 10,000 initial samples generated using Phi-4 +- **Domain-specific curation:** Healthcare-focused questions derived from existing research +- **Deduplication:** Filtered to remove duplicates, resulting in 6,444 unique samples + +### Training Procedure + +#### Fine-tuning Method +- **Technique:** LoRA (Low-Rank Adaptation) +- **Framework:** MLX +- **Base model:** meta-llama/Llama-3.2-3B-Instruct +- **Focus:** Healthcare domain specialization + +#### Training Hyperparameters +- **Fine-tuning approach:** Domain-focused LoRA adaptation +- **Dataset split:** 80% training, 10% validation, 10% testing +- **Training regime:** Optimized for healthcare question-answering performance + +## Evaluation + +### Testing Data & Metrics + +The model was evaluated on: +- **50 healthcare-specific validation questions** +- **Comparative analysis** against base Llama-3.2-3B-Instruct +- **Multi-dimensional assessment** including direct answer capability, response structure, and generation efficiency + +### Key Results + +**Direct Answer Performance:** +- Base model: 6.0% direct answer rate +- Fine-tuned model: 36.0% direct answer rate +- **Improvement: +30.0 percentage points** + +**Response Quality:** +- Enhanced structure with increased use of numbered lists (+18%) +- Improved organization and systematic presentation +- Better alignment with healthcare communication standards + +**Generation Efficiency:** +- Slight increase in generation time (+1.6%) +- Trade-off between response quality and speed +- Overall positive impact on response comprehensiveness + +## Bias, Risks, and Limitations + +### Limitations +- **Not a substitute for professional medical advice** +- **May generate plausible-sounding but incorrect medical information** +- **Limited to English language medical contexts** +- **Training data may not cover all medical specialties equally** +- **Performance may vary across different healthcare subdomains** + +### Recommendations + +- **Always verify medical information** with qualified healthcare professionals +- **Use as a supplementary tool** rather than primary medical resource +- **Implement human oversight** in all healthcare applications +- **Regular updates** needed to maintain medical accuracy as knowledge evolves +- **Consider integration with retrieval-augmented generation (RAG)** for enhanced factual accuracy + +## How to Get Started with the Model + +```python +from transformers import AutoTokenizer, AutoModelForCausalLM + +# Load the model and tokenizer +tokenizer = AutoTokenizer.from_pretrained("adityak74/medfit-llm-3B") +model = AutoModelForCausalLM.from_pretrained("adityak74/medfit-llm-3B") + +# Example usage +prompt = "What are the common symptoms of diabetes?" +inputs = tokenizer.encode(prompt, return_tensors="pt") +outputs = model.generate(inputs, max_length=200, temperature=0.7) +response = tokenizer.decode(outputs[0], skip_special_tokens=True) +print(response) +``` + +## Environmental Impact + +The fine-tuning process utilized efficient LoRA techniques to minimize computational requirements while maximizing performance improvements. This approach reduces the environmental impact compared to full model training while achieving significant domain-specific enhancements. + +## Citation + +**BibTeX:** +```bibtex +@inproceedings{rao2025medfit, + title={MEDFIT-LLM: Medical Enhancements through Domain-Focused Fine Tuning of Small Language Models}, + author={Rao, Aditya Karnam Gururaj and Jaggi, Arjun and Naidu, Sonam}, + booktitle={2025 2nd International Conference on Research Methodologies in Knowledge Management, Artificial Intelligence and Telecommunication Engineering (RMKMATE)}, + year={2025}, + organization={IEEE} +} +``` + +**APA:** +Rao, A. K. G., Jaggi, A., & Naidu, S. (2025). MEDFIT-LLM: Medical Enhancements through Domain-Focused Fine Tuning of Small Language Models. In *2025 2nd International Conference on Research Methodologies in Knowledge Management, Artificial Intelligence and Telecommunication Engineering (RMKMATE)*. IEEE. + +## Glossary + +- **QA:** Question Answering +- **EHR:** Electronic Health Record +- **LoRA:** Low-Rank Adaptation - an efficient fine-tuning technique +- **MLX:** Machine Learning framework used for training +- **Direct Answer Rate:** Percentage of responses that begin with direct answers rather than preambles + +## Model Card Authors + +- **Aditya Karnam Gururaj Rao** (Zefr Inc, LA, USA) +- **Arjun Jaggi** (HCLTech, LA, USA) +- **Sonam Naidu** (LexisNexis, USA) + +## Model Card Contact + +- **Primary Contact:** https://huggingface.co/adityak74 +- **Email:** akarnam37@gmail.com +- **GitHub:** https://github.com/adityak74/medfit-llm + +## Disclaimer + +This model is designed for educational and informational purposes in healthcare contexts. It should not be used as a substitute for professional medical advice, diagnosis, or treatment. Always seek the advice of qualified healthcare providers with questions regarding medical conditions or treatments. \ No newline at end of file