初始化项目，由ModelHub XC社区提供模型
Model: RehanaHasin/mistral-7b-instruct-v0.3-adjuvant-extractor Source: Original Platform
This commit is contained in:
ModelHub XC
245
README.md
Normal file
245
README.md
Normal file
@@ -0,0 +1,245 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- biomedical
|
||||
- infectious-disease
|
||||
- vaccine-adjuvant
|
||||
- information-extraction
|
||||
- evidence-grounded
|
||||
base_model: mistralai/Mistral-7B-Instruct-v0.3
|
||||
---
|
||||
|
||||
# mistral-7b-instruct-v0.3-adjuvant-extractor
|
||||
|
||||
This model is a task-specific fine-tuned version of **Mistral 7B Instruct v0.3** for extracting **vaccine adjuvant concepts** and associated **evidence snippets** from infectious disease biomedical abstracts.
|
||||
|
||||
## Model Summary
|
||||
|
||||
- **Model name:** `mistral-7b-instruct-v0.3-adjuvant-extractor`
|
||||
- **Base model:** `mistralai/Mistral-7B-Instruct-v0.3`
|
||||
- **Fine-tuning method:** LoRA adapter training, merged into full model weights
|
||||
- **Primary task:** Evidence-linked adjuvant extraction from title+abstract text
|
||||
|
||||
## Prompt Used for Inference
|
||||
|
||||
### System prompt
|
||||
|
||||
```text
|
||||
You are a biomedical information extraction assistant.
|
||||
```
|
||||
|
||||
### User instruction template
|
||||
|
||||
```text
|
||||
Extract infectious-disease adjuvants from the text and provide evidence snippets.
|
||||
Return ONLY valid JSON in this format:
|
||||
[{"adjuvant": "<string>", "evidence": "<string>"}, ...]
|
||||
Do not include any extra keys or explanation.
|
||||
```
|
||||
|
||||
### Input format
|
||||
|
||||
```text
|
||||
Title: <paper title>
|
||||
Abstract: <paper abstract>
|
||||
```
|
||||
|
||||
The model receives the user instruction template followed by the title/abstract text.
|
||||
|
||||
## Actual Output Format Target
|
||||
|
||||
The model is prompted to return a JSON array of objects with exactly two keys:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"adjuvant": "<string>",
|
||||
"evidence": "<string>"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
Expected behavior:
|
||||
- Return a JSON array (can be empty: `[]`).
|
||||
- Each object must contain:
|
||||
- `adjuvant`: normalized or near-normalized adjuvant name
|
||||
- `evidence`: supporting text snippet from the same input abstract
|
||||
- No extra keys and no explanatory text outside JSON.
|
||||
|
||||
## Input/Output Example
|
||||
|
||||
### Example Input
|
||||
|
||||
```text
|
||||
Title: Intranasal vaccination study using alum and MPLA adjuvants in a murine influenza model.
|
||||
Abstract: Mice immunized with antigen formulated with alum showed increased IgG titers. A separate group receiving MPLA-adjuvanted vaccine demonstrated stronger IFN-gamma responses and reduced viral load after challenge.
|
||||
```
|
||||
|
||||
### Expected Output
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"adjuvant": "alum",
|
||||
"evidence": "Mice immunized with antigen formulated with alum showed increased IgG titers."
|
||||
},
|
||||
{
|
||||
"adjuvant": "MPLA",
|
||||
"evidence": "A separate group receiving MPLA-adjuvanted vaccine demonstrated stronger IFN-gamma responses and reduced viral load after challenge."
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### Notes on Output Validity
|
||||
|
||||
- Output must be valid JSON.
|
||||
- Output must be a JSON array (use `[]` if no supported adjuvant is found).
|
||||
- Each item should include only `adjuvant` and `evidence`.
|
||||
- Evidence text should come from the provided input abstract.
|
||||
|
||||
## Working Inference Code (Validated)
|
||||
|
||||
```python
|
||||
import torch
|
||||
import json
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
|
||||
repo_id = "RehanaHasin/mistral-7b-instruct-v0.3-adjuvant-extractor"
|
||||
|
||||
SYS_PROMPT = "You are a biomedical information extraction assistant."
|
||||
PROMPT_INSTRUCTION = (
|
||||
"Extract infectious-disease adjuvants from the text and provide evidence snippets.\n"
|
||||
"Return ONLY valid JSON in this format:\n"
|
||||
"[{\"adjuvant\": \"<string>\", \"evidence\": \"<string>\"}, ...]\n"
|
||||
"Do not include any extra keys or explanation."
|
||||
)
|
||||
|
||||
title = "Protective immune response against Streptococcus pyogenes in mice after intranasal vaccination with the fibronectin-binding protein SfbI."
|
||||
abstract = (
|
||||
"Despite the significant impact on human health caused by Streptococcus pyogenes, "
|
||||
"there is currently no vaccine available. Intranasal immunization of mice with either "
|
||||
"SfbI alone or coupled to cholera toxin B subunit (CTB) triggered efficient SfbI-specific responses."
|
||||
)
|
||||
|
||||
user_input = f"{PROMPT_INSTRUCTION}\n\nTitle: {title}\nAbstract: {abstract}"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(repo_id, use_fast=True)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
repo_id,
|
||||
dtype=torch.float16,
|
||||
device_map="auto",
|
||||
)
|
||||
model.eval()
|
||||
|
||||
chat = tokenizer.apply_chat_template(
|
||||
[
|
||||
{"role": "system", "content": SYS_PROMPT},
|
||||
{"role": "user", "content": user_input},
|
||||
],
|
||||
tokenize=False,
|
||||
add_generation_prompt=True,
|
||||
)
|
||||
|
||||
inputs = tokenizer(chat, return_tensors="pt", truncation=True, max_length=1024)
|
||||
inputs = {k: v.to(model.get_input_embeddings().weight.device) for k, v in inputs.items()}
|
||||
|
||||
with torch.no_grad():
|
||||
outputs = model.generate(
|
||||
**inputs,
|
||||
max_new_tokens=200,
|
||||
do_sample=False,
|
||||
pad_token_id=tokenizer.eos_token_id,
|
||||
)
|
||||
|
||||
prediction = tokenizer.decode(
|
||||
outputs[0][inputs["input_ids"].shape[1]:],
|
||||
skip_special_tokens=True
|
||||
).strip()
|
||||
|
||||
print(json.dumps(json.loads(prediction), indent=2, ensure_ascii=False))
|
||||
|
||||
```
|
||||
|
||||
## Intended Use
|
||||
|
||||
This model is intended for research workflows in biomedical literature mining, especially:
|
||||
|
||||
- infectious disease vaccine literature curation
|
||||
- vaccine adjuvant concept extraction
|
||||
- evidence-linked information extraction for downstream manual review
|
||||
|
||||
This model is not intended for clinical decision-making.
|
||||
|
||||
## Training Data and Split Context
|
||||
|
||||
The model was trained on a curated infectious disease adjuvant corpus derived from VIOLIN ecosystem resources.
|
||||
|
||||
- Corpus size used in workflow: 298 abstracts
|
||||
- Fixed split framework used across models:
|
||||
- 256 train
|
||||
- 13 validation
|
||||
- 29 test
|
||||
|
||||
## Training Configuration (Fixed Manuscript Setting)
|
||||
|
||||
- LoRA rank (`r`): 8
|
||||
- Learning rate: `2e-4`
|
||||
- Epochs: `5`
|
||||
- Quantization during fine-tuning: 4-bit NF4 with double quantization
|
||||
- Compute dtype: float16
|
||||
- Per-device batch size and gradient accumulation were configured for stable updates across model families.
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
import torch
|
||||
|
||||
repo_id = "RehanaHasin/mistral-7b-instruct-v0.3-adjuvant-extractor"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(repo_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
repo_id,
|
||||
torch_dtype=torch.float16,
|
||||
device_map="auto",
|
||||
)
|
||||
```
|
||||
|
||||
## Prompting Recommendation
|
||||
|
||||
Use prompts that explicitly request structured JSON output containing only:
|
||||
|
||||
- `adjuvant`
|
||||
- `evidence`
|
||||
|
||||
and restrict extra commentary to reduce parsing errors.
|
||||
|
||||
## Limitations
|
||||
|
||||
- Evaluated on a focused infectious-disease adjuvant corpus; broader-domain generalization is not guaranteed.
|
||||
- Performance depends on abstract quality and terminology variation.
|
||||
- Structured output may still require post-processing and manual validation.
|
||||
|
||||
## Ethical and Safety Notes
|
||||
|
||||
- Outputs can contain extraction errors or unsupported predictions.
|
||||
- Human review is required before downstream knowledge integration.
|
||||
- Not for diagnosis, treatment, or direct patient-care decisions.
|
||||
|
||||
## Reproducibility Resources
|
||||
|
||||
Code, notebooks, and workflow details are available at:
|
||||
https://github.com/hurlab/Infectious-Disease-Adjuvant-LLM-Fine-tuning
|
||||
|
||||
|
||||
## Citation
|
||||
|
||||
If you use this model, please cite the associated manuscript and project repository.
|
||||
|
||||
## Contact
|
||||
|
||||
For questions, please contact hasin.rehana@und.edu.
|
||||
Reference in New Issue
Block a user