初始化项目，由ModelHub XC社区提供模型
Model: CYX1998/Meissa-4B Source: Original Platform
This commit is contained in:
ModelHub XC
161
README.md
Normal file
161
README.md
Normal file
@@ -0,0 +1,161 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: Qwen/Qwen3-VL-4B-Instruct
|
||||
library_name: transformers
|
||||
pipeline_tag: image-text-to-text
|
||||
language:
|
||||
- en
|
||||
tags:
|
||||
- medical
|
||||
- agent
|
||||
- tool-calling
|
||||
- multi-agent
|
||||
- multimodal
|
||||
- qwen3-vl
|
||||
---
|
||||
|
||||
# Meissa-4B: Multi-modal Medical Agentic Intelligence
|
||||
|
||||
<p align="center">
|
||||
<a href="https://arxiv.org/abs/2603.09018"><img src="https://img.shields.io/badge/arXiv-2603.09018-b31b1b.svg" alt="arXiv"></a>
|
||||
<a href="https://huggingface.co/datasets/CYX1998/Meissa-SFT"><img src="https://img.shields.io/badge/HuggingFace-Meissa--SFT-blue" alt="Dataset"></a>
|
||||
<a href="https://github.com/Schuture/Meissa"><img src="https://img.shields.io/badge/GitHub-Meissa-black" alt="GitHub"></a>
|
||||
</p>
|
||||
|
||||
**Meissa-4B** is a lightweight 4B-parameter medical multi-modal LLM with full agentic capability. Instead of relying on proprietary frontier models (GPT, Gemini), Meissa brings tool calling, multi-agent collaboration, and clinical simulation **offline** by distilling structured trajectories from frontier agent systems into a compact vision-language model.
|
||||
|
||||
## Key Features
|
||||
|
||||
- **4 agentic paradigms** in a single model: continuous tool calling, interleaved thinking with images, multi-agent collaboration, and multi-turn clinical simulation
|
||||
- **Offline deployment**: runs entirely locally with vLLM, no API calls needed
|
||||
- **Tool calling**: native `<tool_call>` support via Hermes format, compatible with vLLM's tool-call parser
|
||||
- **Thinking**: built-in `<think>` chain-of-thought reasoning before actions
|
||||
|
||||
## Model Details
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Base model** | [Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct) |
|
||||
| **Architecture** | Qwen3VLForConditionalGeneration |
|
||||
| **Parameters** | 4B |
|
||||
| **Precision** | bfloat16 |
|
||||
| **Training method** | LoRA SFT (rank=32, alpha=64), merged |
|
||||
| **Training data** | 43,210 medical agentic trajectories ([open subset](https://huggingface.co/datasets/CYX1998/Meissa-SFT)) |
|
||||
| **Training framework** | [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) |
|
||||
| **Context length** | 8,192 tokens (training) |
|
||||
| **Tested environment** | transformers 4.57.0, vLLM 0.11.0 |
|
||||
|
||||
## Quickstart
|
||||
|
||||
### Load with Transformers
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForImageTextToText, AutoProcessor
|
||||
import torch
|
||||
|
||||
model = AutoModelForImageTextToText.from_pretrained(
|
||||
"CYX1998/Meissa-4B",
|
||||
torch_dtype=torch.bfloat16,
|
||||
device_map="auto",
|
||||
)
|
||||
processor = AutoProcessor.from_pretrained("CYX1998/Meissa-4B")
|
||||
```
|
||||
|
||||
### Serve with vLLM (Recommended)
|
||||
|
||||
For agentic use cases, serve Meissa with vLLM to enable tool calling:
|
||||
|
||||
```bash
|
||||
python -m vllm.entrypoints.openai.api_server \
|
||||
--model CYX1998/Meissa-4B \
|
||||
--port 8877 \
|
||||
--max-model-len 8192 \
|
||||
--gpu-memory-utilization 0.85 \
|
||||
--dtype bfloat16 \
|
||||
--enable-auto-tool-choice \
|
||||
--tool-call-parser hermes
|
||||
|
||||
# Set the endpoint
|
||||
export OPENAI_BASE_URL="http://127.0.0.1:8877/v1"
|
||||
export OPENAI_API_KEY="dummy"
|
||||
```
|
||||
|
||||
The `--enable-auto-tool-choice --tool-call-parser hermes` flags are **required** for tool calling.
|
||||
|
||||
### Example: Tool Calling
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(base_url="http://127.0.0.1:8877/v1", api_key="dummy")
|
||||
|
||||
tools = [{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "ChestXRayClassifier",
|
||||
"description": "Classify pathologies in a chest X-ray image.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"image_path": {"type": "string", "description": "Path to the chest X-ray image"}
|
||||
},
|
||||
"required": ["image_path"]
|
||||
}
|
||||
}
|
||||
}]
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model="CYX1998/Meissa-4B",
|
||||
messages=[{"role": "user", "content": "Analyze this chest X-ray: /path/to/cxr.jpg"}],
|
||||
tools=tools,
|
||||
)
|
||||
print(response.choices[0].message)
|
||||
```
|
||||
|
||||
## Supported Agentic Frameworks
|
||||
|
||||
| Framework | Description | Tools |
|
||||
|-----------|-------------|-------|
|
||||
| **I: Continuous Tool Calling** | Sequential tool use for radiology analysis | 8 chest X-ray tools (classifier, report generator, VQA, segmentation, etc.) |
|
||||
| **II: Interleaved Thinking with Images** | Iterative visual reasoning with zoom | ZoomInSubfigure, SegmentRegion, Terminate |
|
||||
| **III: Multi-Agent Collaboration** | Multi-agent medical consultation | AssessDifficulty, RecruitExperts, ConsultExperts, FacilitateDebate |
|
||||
| **IV: Clinical Simulation** | Multi-turn doctor-patient interaction | RequestPhysicalExam, RequestTest, Terminate |
|
||||
|
||||
## Training Data
|
||||
|
||||
Trained on 43,210 medical agentic SFT trajectories distilled from Gemini:
|
||||
|
||||
| Framework | Samples | Source Datasets |
|
||||
|-----------|---------|----------------|
|
||||
| I: Continuous Tool Calling | 4,898 | MIMIC-CXR-VQA |
|
||||
| II: Interleaved Thinking | 15,211 | PathVQA, MIMIC-CXR-VQA, SLAKE, VQA-RAD |
|
||||
| III: Multi-Agent Collaboration | 15,427 | MIMIC-CXR-VQA, PathVQA, MedQA, PubMedQA |
|
||||
| IV: Clinical Simulation | 7,674 | MedQA, MIMIC-CXR |
|
||||
|
||||
The open-source subset (25,018 samples) is available at [CYX1998/Meissa-SFT](https://huggingface.co/datasets/CYX1998/Meissa-SFT).
|
||||
|
||||
## Evaluation
|
||||
|
||||
Meissa-4B matches or exceeds GPT-4o and Gemini-3-flash on multiple medical agentic benchmarks while being deployable offline on a single GPU. See [our paper](https://arxiv.org/abs/2603.09018) for full results.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Not for clinical use**: This model is a research prototype and should NOT be used for real clinical decision-making.
|
||||
- **English only**: Trained and evaluated on English medical data only.
|
||||
- **Domain scope**: Primarily trained on radiology, pathology, and general clinical reasoning. Performance on other medical specialties may vary.
|
||||
- **Hallucination**: Like all LLMs, Meissa may generate plausible but incorrect medical information.
|
||||
|
||||
## Citation
|
||||
|
||||
```bibtex
|
||||
@inproceedings{chen2026meissa,
|
||||
title={Meissa: Multi-modal Medical Agentic Intelligence},
|
||||
author={Chen, Yixiong and Bai, Xinyi and Pan, Yue and Zhou, Zongwei and Yuille, Alan},
|
||||
journal={arXiv preprint arXiv:2603.09018},
|
||||
year={2026}
|
||||
}
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
This model is released under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). The base model [Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct) is subject to the [Qwen License](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct/blob/main/LICENSE).
|
||||
Reference in New Issue
Block a user