SeongryongJung/Qwen3-4B-Physics-GRPO-TR
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: SeongryongJung/Qwen3-4B-Physics-GRPO-TR
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-05 02:49:18 +08:00
commit 8100d93918
37 changed files with 313645 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

77
results/hyperparameters.csv Normal file
View File

@@ -0,0 +1,77 @@
section,parameter,value,source
Run identity,Base model,Qwen/Qwen3-4B,queue/script override
Run identity,Dataset,Physics / SciKnowEval physics,run_qwen3_generalization.sh
Run identity,Method,GRPO,run_qwen3_generalization.sh
Run identity,Config,baseline_grpo,run_qwen3_generalization.sh
Run identity,Experiment,qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8,run_qwen3_generalization.sh
Run identity,W&B run,run-20260702_073403-o8ivivjg,wandb
Data,Train file,datasets/sciknoweval/physics/train.parquet,script override
Data,Validation file,datasets/sciknoweval/physics/test.parquet,script override
Data,Train batch size,32,queue/script override
Data,Train max samples,3200,queue/script override
Data,Prompt key,prompt,legacy_data.yaml default
Data,Reward key,data_source,legacy_data.yaml default
Data,Shuffle train data,True,user.yaml / legacy_data.yaml
Data,Validation shuffle,False,legacy_data.yaml default
Data,Filter overlong prompts,True,user.yaml
Data,Prompt truncation,error,legacy_data.yaml default
Data,enable_thinking,false,script override
Schedule,Total training steps,100,queue/script override
Schedule,Total epochs,30,ppo_trainer/user.yaml default
Schedule,Validation before train,False,queue/script override
Schedule,Save frequency,10,queue/script override
Schedule,Validation frequency,10,queue/script override
Sequence,Max prompt length,2048,queue/script override
Sequence,Max response length,8192,queue/script override
Sequence,Max model length,10240,queue/script override
Sequence,Actor max token length per GPU,10240,queue/script override
Rollout,Rollout engine,vllm,user.yaml
Rollout,Rollout dtype,bfloat16,rollout.yaml default
Rollout,Train rollout n,8,queue/script override
Rollout,Train rollout temperature,1.0,script override
Rollout,Train rollout top_p,1.0,script override
Rollout,Train rollout do_sample,True,rollout.yaml default
Rollout,Calculate rollout log probs,True,baseline_grpo.yaml / script override
Rollout,Max num batched tokens,10240,queue/script override
Rollout,vLLM GPU memory utilization,0.8,queue/script override
Rollout,Tensor model parallel size,2,rollout.yaml default
Rollout,Free cache engine,True,rollout.yaml default
Validation,Validation rollout n,16,queue/script override
Validation,Validation temperature,0.6,queue/script override
Validation,Validation top_p,0.95,queue/script override
Validation,Validation do_sample,True,queue/script override
Optimization,Optimizer,AdamW,fsdp optimizer config
Optimization,Learning rate,1e-6,GRPO method override
Optimization,LR scheduler,constant,W&B config
Optimization,LR warmup steps,10,script override
Optimization,Weight decay,0.01,script override
Optimization,Betas,"(0.9, 0.999)",W&B config
Optimization,Gradient clip,1.0,script override
PPO/GRPO,Policy loss mode,vanilla,method override
PPO/GRPO,Advantage estimator,grpo,baseline_grpo.yaml
PPO/GRPO,Normalize GRPO advantages by std,False,baseline_grpo.yaml / script override
PPO/GRPO,PPO epochs,1,W&B config
PPO/GRPO,PPO mini batch size,8,queue/script override
PPO/GRPO,PPO micro batch size per GPU,1,user.yaml
PPO/GRPO,Clip ratio low,0.2,script override
PPO/GRPO,Clip ratio high,0.28,script override
PPO/GRPO,Gamma,1.0,ppo_trainer.yaml default
PPO/GRPO,Lambda,1.0,ppo_trainer.yaml default
PPO/GRPO,Actor KL loss coef,0.0,method override
PPO/GRPO,Use KL in reward,False,ppo_trainer/user.yaml
Rollout correction,Importance sampling mode,token,script override
Rollout correction,IS threshold,2.0,script override
FSDP/System,Actor strategy,fsdp,dp_actor.yaml
FSDP/System,FSDP dtype,bfloat16,W&B config
FSDP/System,FSDP model dtype,fp32,W&B config
FSDP/System,Use torch compile,True,W&B config
FSDP/System,GPUs per node,8,queue/script override
FSDP/System,Nodes,1,user.yaml
FSDP/System,GPU type,NVIDIA H200,wandb-metadata
Checkpoint/Logging,Checkpoint root,checkpoints/datasets/sciknoweval/physics,script override
Checkpoint/Logging,Latest checkpointed iteration,100,latest_checkpointed_iteration.txt
Checkpoint/Logging,Max actor checkpoints to keep,1,user.yaml
Checkpoint/Logging,Logger,"console, wandb",ppo_trainer.yaml
Checkpoint/Logging,W&B entity,seongryongjung-chung-ang-university,environment
Checkpoint/Logging,W&B project,SDPO-root,user.yaml project_name
Checkpoint/Logging,W&B group,QWEN3-GRPO-generalization,method override
1 section parameter value source
2 Run identity Base model Qwen/Qwen3-4B queue/script override
3 Run identity Dataset Physics / SciKnowEval physics run_qwen3_generalization.sh
4 Run identity Method GRPO run_qwen3_generalization.sh
5 Run identity Config baseline_grpo run_qwen3_generalization.sh
6 Run identity Experiment qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 run_qwen3_generalization.sh
7 Run identity W&B run run-20260702_073403-o8ivivjg wandb
8 Data Train file datasets/sciknoweval/physics/train.parquet script override
9 Data Validation file datasets/sciknoweval/physics/test.parquet script override
10 Data Train batch size 32 queue/script override
11 Data Train max samples 3200 queue/script override
12 Data Prompt key prompt legacy_data.yaml default
13 Data Reward key data_source legacy_data.yaml default
14 Data Shuffle train data True user.yaml / legacy_data.yaml
15 Data Validation shuffle False legacy_data.yaml default
16 Data Filter overlong prompts True user.yaml
17 Data Prompt truncation error legacy_data.yaml default
18 Data enable_thinking false script override
19 Schedule Total training steps 100 queue/script override
20 Schedule Total epochs 30 ppo_trainer/user.yaml default
21 Schedule Validation before train False queue/script override
22 Schedule Save frequency 10 queue/script override
23 Schedule Validation frequency 10 queue/script override
24 Sequence Max prompt length 2048 queue/script override
25 Sequence Max response length 8192 queue/script override
26 Sequence Max model length 10240 queue/script override
27 Sequence Actor max token length per GPU 10240 queue/script override
28 Rollout Rollout engine vllm user.yaml
29 Rollout Rollout dtype bfloat16 rollout.yaml default
30 Rollout Train rollout n 8 queue/script override
31 Rollout Train rollout temperature 1.0 script override
32 Rollout Train rollout top_p 1.0 script override
33 Rollout Train rollout do_sample True rollout.yaml default
34 Rollout Calculate rollout log probs True baseline_grpo.yaml / script override
35 Rollout Max num batched tokens 10240 queue/script override
36 Rollout vLLM GPU memory utilization 0.8 queue/script override
37 Rollout Tensor model parallel size 2 rollout.yaml default
38 Rollout Free cache engine True rollout.yaml default
39 Validation Validation rollout n 16 queue/script override
40 Validation Validation temperature 0.6 queue/script override
41 Validation Validation top_p 0.95 queue/script override
42 Validation Validation do_sample True queue/script override
43 Optimization Optimizer AdamW fsdp optimizer config
44 Optimization Learning rate 1e-6 GRPO method override
45 Optimization LR scheduler constant W&B config
46 Optimization LR warmup steps 10 script override
47 Optimization Weight decay 0.01 script override
48 Optimization Betas (0.9, 0.999) W&B config
49 Optimization Gradient clip 1.0 script override
50 PPO/GRPO Policy loss mode vanilla method override
51 PPO/GRPO Advantage estimator grpo baseline_grpo.yaml
52 PPO/GRPO Normalize GRPO advantages by std False baseline_grpo.yaml / script override
53 PPO/GRPO PPO epochs 1 W&B config
54 PPO/GRPO PPO mini batch size 8 queue/script override
55 PPO/GRPO PPO micro batch size per GPU 1 user.yaml
56 PPO/GRPO Clip ratio low 0.2 script override
57 PPO/GRPO Clip ratio high 0.28 script override
58 PPO/GRPO Gamma 1.0 ppo_trainer.yaml default
59 PPO/GRPO Lambda 1.0 ppo_trainer.yaml default
60 PPO/GRPO Actor KL loss coef 0.0 method override
61 PPO/GRPO Use KL in reward False ppo_trainer/user.yaml
62 Rollout correction Importance sampling mode token script override
63 Rollout correction IS threshold 2.0 script override
64 FSDP/System Actor strategy fsdp dp_actor.yaml
65 FSDP/System FSDP dtype bfloat16 W&B config
66 FSDP/System FSDP model dtype fp32 W&B config
67 FSDP/System Use torch compile True W&B config
68 FSDP/System GPUs per node 8 queue/script override
69 FSDP/System Nodes 1 user.yaml
70 FSDP/System GPU type NVIDIA H200 wandb-metadata
71 Checkpoint/Logging Checkpoint root checkpoints/datasets/sciknoweval/physics script override
72 Checkpoint/Logging Latest checkpointed iteration 100 latest_checkpointed_iteration.txt
73 Checkpoint/Logging Max actor checkpoints to keep 1 user.yaml
74 Checkpoint/Logging Logger console, wandb ppo_trainer.yaml
75 Checkpoint/Logging W&B entity seongryongjung-chung-ang-university environment
76 Checkpoint/Logging W&B project SDPO-root user.yaml project_name
77 Checkpoint/Logging W&B group QWEN3-GRPO-generalization method override