Qwen3-8B-Physics-GRPO-TR/README.md

license, library_name, pipeline_tag, tags, base_model

license	library_name	pipeline_tag	tags	base_model
apache-2.0	transformers	text-generation	qwen3 reinforcement-learning grpo text-generation	Qwen/Qwen3-8B

Qwen3-8B-Physics-GRPO-TR

This repository contains the Qwen3-8B physics GRPO batch-size-32 run. The repository name uses the project GRPO-TR naming convention, but the actual training method for this checkpoint is GRPO.

The repository root contains the best validation checkpoint, selected by validation mean@16. checkpoints/last/ contains the final checkpoint.

Performance

Dataset	Method	Base model	Train batch size	Best val mean@16	Best checkpoint	Final val mean@16	Final checkpoint
Physics / SciKnowEval physics	GRPO	Qwen3-8B	32	72.97%	100	72.97%	100

Validation Mean@16

step	val_mean16	percent
10	0.583593750000	58.36%
20	0.599218750000	59.92%
30	0.604687500000	60.47%
40	0.621093750000	62.11%
50	0.653906250000	65.39%
60	0.671875000000	67.19%
70	0.681250000000	68.12%
80	0.712500000000	71.25%
90	0.722656250000	72.27%
100	0.729687500000	72.97%

Detailed Training Hyperparameters

Section	Parameter	Value	Source
Run identity	Base model	Qwen/Qwen3-8B	queue/script override
Run identity	Dataset	Physics / SciKnowEval physics	run_qwen3_generalization.sh
Run identity	Method	GRPO	run_qwen3_generalization.sh
Run identity	Config	baseline_grpo	run_qwen3_generalization.sh
Run identity	Experiment	qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8	run_qwen3_generalization.sh
Run identity	W&B run	run-20260703_032040-6ig3l55l	wandb
Data	Train file	datasets/sciknoweval/physics/train.parquet	script override
Data	Validation file	datasets/sciknoweval/physics/test.parquet	script override
Data	Train batch size	32	queue/script override
Data	Train max samples	3200	queue/script override
Schedule	Total training steps	100	queue/script override
Schedule	Validation before train	False	queue/script override
Schedule	Save frequency	10	queue/script override
Schedule	Validation frequency	10	queue/script override
Sequence	Max prompt length	2048	queue/script override
Sequence	Max response length	8192	queue/script override
Sequence	Max model length	10240	queue/script override
Rollout	Train rollout n	8	queue/script override
Rollout	Validation rollout n	16	queue/script override
Rollout	vLLM GPU memory utilization	0.8	queue/script override
Optimization	Learning rate	1e-6	GRPO method override
Optimization	Weight decay	0.01	script override
PPO/GRPO	PPO mini batch size	8	queue/script override
PPO/GRPO	Normalize GRPO advantages by std	False	baseline_grpo.yaml / script override
Rollout correction	Importance sampling mode	token	script override
Rollout correction	IS threshold	2.0	script override
Checkpoint/Logging	Checkpoint root	checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8	script override
Checkpoint/Logging	Latest checkpointed iteration	100	latest_checkpointed_iteration.txt
Checkpoint/Logging	External actor archive	checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/_actor_archive	preserve_actor_checkpoints.py
Checkpoint/Logging	Logger	console, wandb	ppo_trainer.yaml
PPO/GRPO	Policy loss mode	vanilla	method override
PPO/GRPO	Actor KL loss coef	0.0	method override

Raw result and artifact files:

• results/validation_mean16.csv
• results/training_scores.csv
• results/hyperparameters.csv
• results/training_score.png
• results/training_score.svg
• artifacts/output.log
• artifacts/queue.log

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "SeongryongJung/Qwen3-8B-Physics-GRPO-TR"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

Source

• Checkpoint: checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8
• Root actor checkpoint: checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100/actor
• Last actor checkpoint: checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-8B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100/actor
• W&B run: run-20260703_032040-6ig3l55l
• Queue log: artifacts/queue.log