141 lines
3.3 KiB
Markdown
141 lines
3.3 KiB
Markdown
---
|
|
license: apache-2.0
|
|
base_model: Qwen/Qwen3-8B
|
|
tags:
|
|
- trialpanorama
|
|
- clinical-trials
|
|
- study-search
|
|
- rlvr
|
|
- reinforcement-learning
|
|
- qwen3
|
|
language:
|
|
- en
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# Qwen-3-8B-TP
|
|
|
|
This model is fine-tuned from [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) by using [TrialPanorama dataset](https://huggingface.co/datasets/TrialPanorama/Dataset) for clinical trials.
|
|
|
|
## Model Details
|
|
|
|
- **Base Model**: Qwen/Qwen3-8B
|
|
- **Fine-tuning Method**: Two-stage training
|
|
- Stage 1: Supervised Fine-Tuning (SFT) for knowledge injection
|
|
- Stage 2: RLVR (Reinforcement Learning with Verifiable Reward)
|
|
|
|
## Usage
|
|
|
|
### Basic Usage with Transformers
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
import torch
|
|
|
|
# Load model and tokenizer
|
|
model_name = "TrialPanorama/Qwen-3-8B-TP"
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
model_name,
|
|
torch_dtype=torch.bfloat16,
|
|
device_map="auto"
|
|
)
|
|
|
|
# Prepare input (a toy example)
|
|
prompt = """Given the following clinical trial information, search for relevant studies:
|
|
|
|
[Input Information]
|
|
|
|
Please provide relevant clinical trials and reasoning."""
|
|
|
|
# Generate response
|
|
messages = [
|
|
{"role": "user", "content": prompt}
|
|
]
|
|
text = tokenizer.apply_chat_template(
|
|
messages,
|
|
tokenize=False,
|
|
add_generation_prompt=True,
|
|
enable_thinking=True
|
|
)
|
|
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
|
|
|
outputs = model.generate(
|
|
**model_inputs,
|
|
max_new_tokens=512,
|
|
temperature=0.6,
|
|
top_p=0.95,
|
|
do_sample=True
|
|
)
|
|
|
|
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
|
print(response)
|
|
```
|
|
|
|
### Usage with vLLM (Recommended for Production)
|
|
|
|
```python
|
|
from vllm import LLM, SamplingParams
|
|
|
|
# Initialize vLLM
|
|
llm = LLM(
|
|
model="TrialPanorama/Qwen-3-8B-TP",
|
|
tensor_parallel_size=1,
|
|
dtype="bfloat16",
|
|
enable_reasoning=True,
|
|
reasoning_parser="deepseek_r1"
|
|
)
|
|
|
|
# Set sampling parameters
|
|
sampling_params = SamplingParams(
|
|
temperature=0.6,
|
|
top_p=0.95,
|
|
max_tokens=512
|
|
)
|
|
|
|
# Generate
|
|
prompts = ["Your study search prompt here"]
|
|
outputs = llm.generate(prompts, sampling_params)
|
|
|
|
for output in outputs:
|
|
print(output.outputs[0].text)
|
|
```
|
|
|
|
### Usage with SGLang
|
|
|
|
```python
|
|
from sglang import Runtime, Engine, ModelConfig, SamplingParams
|
|
|
|
# Initialize SGLang
|
|
runtime = Runtime(
|
|
model_path="TrialPanorama/Qwen-3-8B-TP",
|
|
reasoning_parser="qwen3"
|
|
)
|
|
|
|
# Set sampling parameters
|
|
sampling_params = SamplingParams(
|
|
temperature=0.6,
|
|
top_p=0.95,
|
|
max_new_tokens=512
|
|
)
|
|
|
|
# Generate
|
|
prompts = ["Your study search prompt here"]
|
|
outputs = runtime.generate(prompts, sampling_params)
|
|
|
|
for output in outputs:
|
|
print(output.text)
|
|
```
|
|
|
|
## Citation
|
|
|
|
```bibtex
|
|
@article{wang2025trialpanorama,
|
|
title = {Developing Large Language Models for Clinical Research Using One Million Clinical Trials},
|
|
author = {Wang, Zifeng and Lin, Jiacheng and Jin, Qiao and Gao, Junyi and Pradeepkumar, Jathurshan and Jiang, Pengcheng and Lu, Zhiyong and Sun, Jimeng},
|
|
journal = {arXiv preprint arXiv:2505.16097},
|
|
year = {2025},
|
|
url = {https://arxiv.org/abs/2505.16097}
|
|
}
|
|
```
|