初始化项目,由ModelHub XC社区提供模型
Model: FreedomIntelligence/HuatuoGPT-3-7B-Pangu Source: Original Platform
This commit is contained in:
130
README.md
Normal file
130
README.md
Normal file
@@ -0,0 +1,130 @@
|
||||
---
|
||||
library_name: transformers
|
||||
license: apache-2.0
|
||||
pipeline_tag: text-generation
|
||||
language:
|
||||
- en
|
||||
- zh
|
||||
base_model:
|
||||
- FreedomIntelligence/openPangu-Embedded-7B
|
||||
tags:
|
||||
- medical
|
||||
- reasoning
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
<h1>🩺 HuatuoGPT-3-7B-Pangu</h1>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<a href="https://github.com/FreedomIntelligence/HuatuoGPT-3" target="_blank">🏠 GitHub</a>
|
||||
|
|
||||
<a href="" target="_blank">📄 Paper</a>
|
||||
</div>
|
||||
|
||||
# <span>Introduction</span>
|
||||
|
||||
**HuatuoGPT-3** is an open-source medical LLM trained with **SeedRL**, an RL-only domain adaptation paradigm that transforms a base model into a medical expert in a single RL stage.
|
||||
|
||||
**HuatuoGPT-3-7B-Pangu** is the Pangu-based variant in the HuatuoGPT-3 series. Different from the Qwen-based versions, it is built on **FreedomIntelligence/openPangu-Embedded-7B** and trained with **Ascend NPUs**.
|
||||
|
||||
For more information, visit our GitHub repository:
|
||||
[https://github.com/FreedomIntelligence/HuatuoGPT-3](https://github.com/FreedomIntelligence/HuatuoGPT-3)
|
||||
|
||||
> [!IMPORTANT]
|
||||
> **HuatuoGPT-3-7B-Pangu is set to thinking mode by default.** Since it is based on openPangu, the generated reasoning content is placed between `[unused16]` and `[unused17]`, and the final response starts after `[unused17]`.
|
||||
|
||||
# <span>Model Info</span>
|
||||
|
||||
| Model | Description | Backbone | Link |
|
||||
| --- | --- | --- | --- |
|
||||
| **HuatuoGPT-3-32B** | 32B medical LLM trained with SeedRL | Qwen3-32B | [HF Link](https://huggingface.co/FreedomIntelligence/HuatuoGPT-3-32B) |
|
||||
| **HuatuoGPT-3-8B** | 8B medical LLM trained with SeedRL | Qwen3-8B-Base | [HF Link](https://huggingface.co/FreedomIntelligence/HuatuoGPT-3-8B) |
|
||||
| **HuatuoGPT-3-7B-Pangu** | 7B medical LLM trained with SeedRL | openPangu-Embedded-7B | [HF Link](https://huggingface.co/FreedomIntelligence/HuatuoGPT-3-7B-Pangu) |
|
||||
|
||||
# <span>Usage</span>
|
||||
|
||||
You can use HuatuoGPT-3-7B-Pangu in the same way as `FreedomIntelligence/openPangu-Embedded-7B`.
|
||||
|
||||
When using this model, please note that the Pangu backbone requires `trust_remote_code=True` in Transformers, and `--trust_remote_code` when serving with vLLM.
|
||||
|
||||
- Direct inference:
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_name = "FreedomIntelligence/HuatuoGPT-3-7B-Pangu"
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(
|
||||
model_name,
|
||||
use_fast=False,
|
||||
trust_remote_code=True
|
||||
)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_name,
|
||||
trust_remote_code=True,
|
||||
torch_dtype="auto",
|
||||
device_map="auto"
|
||||
)
|
||||
|
||||
messages = [
|
||||
{"role": "user", "content": "What are the common causes of chest pain?"}
|
||||
]
|
||||
|
||||
text = tokenizer.apply_chat_template(
|
||||
messages,
|
||||
tokenize=False,
|
||||
add_generation_prompt=True
|
||||
)
|
||||
|
||||
inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
||||
outputs = model.generate(
|
||||
**inputs,
|
||||
max_new_tokens=4096,
|
||||
eos_token_id=45892,
|
||||
return_dict_in_generate=True
|
||||
)
|
||||
|
||||
input_length = inputs.input_ids.shape[1]
|
||||
generated_tokens = outputs.sequences[:, input_length:]
|
||||
output_text = tokenizer.decode(generated_tokens[0])
|
||||
|
||||
thinking_content = output_text.split("[unused17]")[0].split("[unused16]")[-1].strip()
|
||||
content = output_text.split("[unused17]")[-1].split("[unused10]")[0].strip()
|
||||
|
||||
print("thinking content:", thinking_content)
|
||||
print("content:", content)
|
||||
```
|
||||
|
||||
- You can also serve the model with vLLM. Make sure to include `--trust_remote_code`:
|
||||
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0 \
|
||||
vllm serve FreedomIntelligence/HuatuoGPT-3-7B-Pangu \
|
||||
--served-model-name HuatuoGPT-3-7B-Pangu \
|
||||
--trust_remote_code \
|
||||
--port 8000
|
||||
```
|
||||
|
||||
Or:
|
||||
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0 \
|
||||
python -m vllm.entrypoints.openai.api_server \
|
||||
--model FreedomIntelligence/HuatuoGPT-3-7B-Pangu \
|
||||
--served-model-name HuatuoGPT-3-7B-Pangu \
|
||||
--trust_remote_code \
|
||||
--port 8000
|
||||
```
|
||||
|
||||
|
||||
# <span>📖 Citation</span>
|
||||
|
||||
```bibtex
|
||||
@article{huatuogpt3,
|
||||
title={HuatuoGPT-3: RL-Only Domain Adaptation from Base Models via Off-Policy Seeding},
|
||||
author={Coming soon},
|
||||
journal={arXiv preprint},
|
||||
year={2026}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user