58 lines
1.8 KiB
Markdown
58 lines
1.8 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
base_model: jiosephlee/Intern-S1-mini-lm
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- chemistry
|
||
|
|
- drug-discovery
|
||
|
|
- admet
|
||
|
|
- assay-transfer
|
||
|
|
- smiles
|
||
|
|
library_name: transformers
|
||
|
|
---
|
||
|
|
|
||
|
|
# assay-transfer-tool
|
||
|
|
|
||
|
|
A full-parameter SFT of the `Qwen3ForCausalLM` Intern-S1-mini language backbone
|
||
|
|
(`jiosephlee/Intern-S1-mini-lm`) for **binary assay-transfer prediction** on
|
||
|
|
small molecules. Given a molecule (SMILES) and a paired-choice prompt, the model
|
||
|
|
answers with `(A)` or `(B)`.
|
||
|
|
|
||
|
|
The model uses the Intern SMILES-aware tokenizer, so load with
|
||
|
|
`trust_remote_code=True`.
|
||
|
|
|
||
|
|
## Training
|
||
|
|
|
||
|
|
- **Base model:** `jiosephlee/Intern-S1-mini-lm` (Qwen3ForCausalLM, vocab 153216)
|
||
|
|
- **Dataset:** `jiosephlee/assay-transfer-intern` (158,429 train rows)
|
||
|
|
- **Regime:** full-parameter BF16, packing + padding-free, gradient checkpointing,
|
||
|
|
Liger fused linear cross-entropy, `paged_adamw_8bit`
|
||
|
|
- **Chat template:** enabled, thinking disabled; completions are `(A)` / `(B)`
|
||
|
|
- **LR:** 2e-5 base, with SMILES input-embedding rows trained at 1.5x
|
||
|
|
- **Max length:** 4096, 1 epoch
|
||
|
|
|
||
|
|
This checkpoint is the best-validation snapshot (step 50), promoted by the
|
||
|
|
assay-transfer callback on binary macro-F1.
|
||
|
|
|
||
|
|
## Validation metrics (source_value split, n=1703)
|
||
|
|
|
||
|
|
| Metric | Value |
|
||
|
|
| --- | --- |
|
||
|
|
| Binary macro-F1 | 0.6612 |
|
||
|
|
| Binary accuracy | 0.7046 |
|
||
|
|
| Parse rate | 1.00 |
|
||
|
|
| assay_concept group-avg macro-F1 | 0.6321 |
|
||
|
|
| tanimoto_bucket group-avg macro-F1 | 0.6316 |
|
||
|
|
|
||
|
|
Per-assay macro-F1: Fa 0.532, Fg 0.593, Fh 0.704, oral_bioavailability 0.600,
|
||
|
|
oral_exposure 0.732. Per-Tanimoto-bucket macro-F1: high 0.664, low 0.600.
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
tok = AutoTokenizer.from_pretrained("jiosephlee/assay-transfer-tool", trust_remote_code=True)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained("jiosephlee/assay-transfer-tool", torch_dtype="bfloat16")
|
||
|
|
```
|