89 lines
2.4 KiB
Markdown
89 lines
2.4 KiB
Markdown
---
|
|
license: apache-2.0
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- qwen3
|
|
- reinforcement-learning
|
|
- grpo
|
|
- text-generation
|
|
- text-generation-inference
|
|
base_model: Qwen/Qwen3-4B
|
|
---
|
|
|
|
# Qwen3-4B-Chemical-GRPO-TR
|
|
|
|
This repository contains the Qwen3-4B `Chemical` `GRPO` batch-size-32 run. The repository name uses the requested `GRPO-TR` suffix, but the actual training method for this checkpoint is GRPO.
|
|
|
|
The repository root contains the best validation checkpoint (`global_step_100`), selected by validation `mean@16`. `checkpoints/last/` contains the final checkpoint. For this run, best and final are the same checkpoint.
|
|
|
|
## Performance
|
|
|
|
| Dataset | Method | Base model | Train batch size | Best val mean@16 | Best checkpoint | Final val mean@16 | Final checkpoint |
|
|
|---|---|---|---:|---:|---:|---:|---:|
|
|
| Chemical | GRPO | Qwen3-4B | 32 | 68.81% | 100 | 68.81% | 100 |
|
|
|
|
|
|
|
|
Raw result files:
|
|
|
|
- `results/validation_mean16.csv`
|
|
- `results/training_scores.csv`
|
|
- `artifacts/config.yaml`
|
|
- `artifacts/wandb-summary.json`
|
|
- `artifacts/wandb-metadata.json`
|
|
- `artifacts/output.log`
|
|
|
|
## Validation Mean@16
|
|
|
|
| step | val_mean16 |
|
|
|---:|---:|
|
|
| 10 | 0.439880952381 |
|
|
| 20 | 0.505952380952 |
|
|
| 30 | 0.552678571429 |
|
|
| 40 | 0.590476190476 |
|
|
| 50 | 0.631845238095 |
|
|
| 60 | 0.648214285714 |
|
|
| 70 | 0.655952380952 |
|
|
| 80 | 0.665773809524 |
|
|
| 90 | 0.678869047619 |
|
|
| 100 | 0.688095238095 |
|
|
|
|
## Training Hyperparameters
|
|
|
|
| Hyperparameter | Value |
|
|
|---|---|
|
|
| Base model | `Qwen/Qwen3-4B` |
|
|
| Method | `GRPO` |
|
|
| Train batch size | `32` |
|
|
| Train max samples | `3200` |
|
|
| Total training steps | `100` |
|
|
| Save frequency | `10` |
|
|
| Test frequency | `10` |
|
|
| Rollout samples per prompt | `8` |
|
|
| Validation samples per prompt | `16` |
|
|
| Learning rate | `1e-6` |
|
|
| vLLM GPU memory utilization | `0.8` |
|
|
| Parsed train score rows | `100` |
|
|
|
|
## Usage
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
repo_id = "SeongryongJung/Qwen3-4B-Chemical-GRPO-TR"
|
|
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
repo_id,
|
|
torch_dtype="auto",
|
|
device_map="auto",
|
|
trust_remote_code=True,
|
|
)
|
|
```
|
|
|
|
## Source
|
|
|
|
- Checkpoint: `checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8`
|
|
- W&B run: `run-20260702_050732-0ozhfamy`
|
|
- Full queue log: `artifacts/queue.log`
|