54 lines
2.2 KiB
Markdown
54 lines
2.2 KiB
Markdown
---
|
|
license: apache-2.0
|
|
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
|
|
tags:
|
|
- text-generation
|
|
- lora
|
|
- peft
|
|
- merged
|
|
- medical
|
|
- fine-tuned
|
|
datasets:
|
|
- BI55/MedText
|
|
language:
|
|
- en
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# TinyLlama-1.1B — Medical QLoRA (Merged)
|
|
|
|
A QLoRA fine-tuned variant of [TinyLlama-1.1B-intermediate-step-1431k-3T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T), trained on medical question-answering data. The LoRA adapter has been merged into the base weights, so this repo is a standalone model — no separate adapter loading required.
|
|
|
|
## ⚠️ Intended use
|
|
|
|
This model was trained as part of a **GenAI training lab** to demonstrate the fine-tuning → merge → deploy workflow. It is **not validated for real clinical or medical use** and should not be used to inform actual medical decisions. Treat its outputs as educational examples of what a small fine-tuned LLM can produce, not medical advice.
|
|
|
|
## Training details
|
|
|
|
- **Base model:** `TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T`
|
|
- **Method:** QLoRA (4-bit NF4 quantization during training) via `peft` + `trl.SFTTrainer`
|
|
- **LoRA config:** `r=8`, `lora_alpha=16`
|
|
- **Dataset:** [`BI55/MedText`](https://huggingface.co/datasets/BI55/MedText) — medical case/question-answer pairs
|
|
- **Steps:** 400
|
|
- **Merge:** adapter merged into the base model via `PeftModel.merge_and_unload()`, then saved as a full standalone model
|
|
|
|
## How to use
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
|
|
|
|
repo_id = "tweijie/tinyllama-medical-merged"
|
|
|
|
model = AutoModelForCausalLM.from_pretrained(repo_id)
|
|
tokenizer = AutoTokenizer.from_pretrained(repo_id)
|
|
|
|
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, max_length=200)
|
|
prompt = "<s>[INST] A patient presents with restless leg sensations at night. What could be the cause? [/INST]"
|
|
print(pipe(prompt)[0]["generated_text"])
|
|
|
|
|
|
Limitations
|
|
1.1B parameters — small model, prone to hallucination and factual errors, especially on medical content.
|
|
Fine-tuned on a single narrow dataset (400 training steps); not broadly evaluated.
|
|
No safety/alignment tuning beyond the base model's own training. |