bio-posttrain-qwen3-4b-cpt/README.md

license, base_model, tags, library_name

license	base_model	tags	library_name
apache-2.0	Qwen/Qwen3-4B	biology bio-posttrain cpt	transformers

Bio-posttrain Qwen3-4B CPT

Continued pre-training (CPT) checkpoint from How Post-Training Shapes Biological Reasoning Models.

Model details

• Base model: Qwen/Qwen3-4B
• Stage: CPT (text-only omics corpus)
• Training: lr=1e-5, gradient accumulation=64, final CPT checkpoint

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("mims-harvard/bio-posttrain-qwen3-4b-cpt", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("mims-harvard/bio-posttrain-qwen3-4b-cpt", trust_remote_code=True)

Collection

Part of the Bio-posttrain collection on Hugging Face.

Citation

@article{bio_posttrain_2026,
  title={How Post-Training Shapes Biological Reasoning Models},
  author={...},
  year={2026}
}