68 lines
3.7 KiB
Markdown
68 lines
3.7 KiB
Markdown
|
---
|
|||
|
|
library_name: transformers
|
|||
|
|
tags:
|
|||
|
|
- biology
|
|||
|
|
- protein-language-model
|
|||
|
|
- protein-generation
|
|||
|
|
- causal-lm
|
|||
|
|
- mixture-of-experts
|
|||
|
|
- transformers
|
|||
|
|
- dpo
|
|||
|
|
- alignment
|
|||
|
|
- protein-design
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# Model Card for ProtGPT3-10B-dpo
|
|||
|
|
|
|||
|
|
## Model Description
|
|||
|
|
ProtGPT3-10B-dpo is the DPO-aligned version of [ProtGPT3-10B](https://huggingface.co/AI4PD/ProtGPT3-10B). It is part of the [ProtGPT3 family](https://huggingface.co/collections/AI4PD/protgpt3-family), an open-source suite of promptable and aligned protein language models for protein design.
|
|||
|
|
|
|||
|
|
ProtGPT3-10B-dpo was further aligned with Direct Preference Optimization (DPO) to improve generation quality. The alignment procedure shifts the model toward protein sequences with higher predicted structural confidence and reduced low-complexity content, while preserving sequence diversity.
|
|||
|
|
For protein generation, each model -dpo version is recommended over the base model.
|
|||
|
|
|
|||
|
|
For more info and guidance on how to generate sequences with ProtGPT3-10B-dpo check out the extensive description provided in [ProtGPT3-1.3B](https://huggingface.co/AI4PD/ProtGPT3-1.3B), just replacing the model name (i.e., `model_name=AI4PD/ProtGPT3-10B-dpo`).
|
|||
|
|
|
|||
|
|
## Uses
|
|||
|
|
|
|||
|
|
### Direct Use
|
|||
|
|
|
|||
|
|
ProtGPT3-10B-dpo can be used for single-sequence autoregressive protein generation. Users can generate protein sequences unconditionally or condition generation on an amino-acid prefix.
|
|||
|
|
|
|||
|
|
Compared with the base ProtGPT3-10B checkpoint, this DPO-aligned model is intended for users who want generations biased toward higher-complexity sequences with improved predicted structural confidence.
|
|||
|
|
|
|||
|
|
### Downstream Use
|
|||
|
|
|
|||
|
|
The model may be used in protein design workflows, computational screening pipelines, protein variant generation, and candidate sequence proposal. Generated sequences can be further evaluated with structure prediction, sequence-complexity filters, solubility filters, fitness predictors, or experimental validation.
|
|||
|
|
|
|||
|
|
### Out-of-Scope Use
|
|||
|
|
|
|||
|
|
The model should not be used as the sole basis for experimental, clinical, environmental, or safety-critical decisions. Generated sequences require downstream computational and experimental validation. The model is not guaranteed to generate functional, soluble, safe, synthesizable, or experimentally successful proteins.
|
|||
|
|
|
|||
|
|
The model should not be used for irresponsible or harmful biological design applications.
|
|||
|
|
|
|||
|
|
## Bias, Risks, and Limitations
|
|||
|
|
|
|||
|
|
ProtGPT3-10B-dpo learns from public protein sequence datasets and may reproduce biases present in those datasets. Although DPO alignment reduces low-complexity generations and improves generation quality according to the alignment objectives (pLDDT and reduction of lcr, as a binary objective, see main manuscript), generated sequences may still be nonfunctional, unstable, insoluble, repetitive, biologically implausible, or unsuitable for a user’s intended application.
|
|||
|
|
|
|||
|
|
The DPO alignment objective uses predicted structural confidence and low-complexity filtering as proxy objectives. These proxies do not guarantee biological function, experimental success, safety, solubility, or manufacturability.
|
|||
|
|
|
|||
|
|
As with other generative protein models, ProtGPT3-10B-dpo may present dual-use risks if applied irresponsibly.
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Citation
|
|||
|
|
|
|||
|
|
**BibTeX:**
|
|||
|
|
|
|||
|
|
```bibtex
|
|||
|
|
@article{protgpt3,
|
|||
|
|
title={ProtGPT3: an Open-source family of Promptable and Aligned Protein Language Models},
|
|||
|
|
author={Anonymous Authors},
|
|||
|
|
year={2026}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
## More Information
|
|||
|
|
For more info and guidance on how to generate sequences with ProtGPT3-10B-dpo check out the extensive description provided in [ProtGPT3-1.3B](https://huggingface.co/AI4PD/ProtGPT3-1.3B), just replacing the model name (i.e., `model_name=AI4PD/ProtGPT3-10B-dpo`).
|
|||
|
|
|
|||
|
|
All models and code are released through the Hugging Face ecosystem and accompanying code repository.
|