---
license: apache-2.0
base_model: Qwen/Qwen3-4B
tags:
  - biology
  - bio-posttrain
  - cpt
library_name: transformers
---

# Bio-posttrain Qwen3-4B CPT

Continued pre-training (CPT) checkpoint from [How Post-Training Shapes Biological Reasoning Models](https://huggingface.co/collections/mims-harvard/bio-posttrain).

## Model details

- **Base model:** `Qwen/Qwen3-4B`
- **Stage:** CPT (text-only omics corpus)
- **Training:** lr=1e-5, gradient accumulation=64, final CPT checkpoint

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("mims-harvard/bio-posttrain-qwen3-4b-cpt", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("mims-harvard/bio-posttrain-qwen3-4b-cpt", trust_remote_code=True)
```

## Collection

Part of the [Bio-posttrain](https://huggingface.co/collections/mims-harvard/bio-posttrain) collection on Hugging Face.

## Citation

```bibtex
@article{bio_posttrain_2026,
  title={How Post-Training Shapes Biological Reasoning Models},
  author={...},
  year={2026}
}
```