SeongryongJung/Qwen3-4B-Physics-GRPO-TR
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: SeongryongJung/Qwen3-4B-Physics-GRPO-TR
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-05 02:49:18 +08:00
commit 8100d93918
37 changed files with 313645 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

77
results/hyperparameters.csv Normal file
View File

@@ -0,0 +1,77 @@
section,parameter,value,source
Run identity,Base model,Qwen/Qwen3-4B,queue/script override
Run identity,Dataset,Physics / SciKnowEval physics,run_qwen3_generalization.sh
Run identity,Method,GRPO,run_qwen3_generalization.sh
Run identity,Config,baseline_grpo,run_qwen3_generalization.sh
Run identity,Experiment,qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8,run_qwen3_generalization.sh
Run identity,W&B run,run-20260702_073403-o8ivivjg,wandb
Data,Train file,datasets/sciknoweval/physics/train.parquet,script override
Data,Validation file,datasets/sciknoweval/physics/test.parquet,script override
Data,Train batch size,32,queue/script override
Data,Train max samples,3200,queue/script override
Data,Prompt key,prompt,legacy_data.yaml default
Data,Reward key,data_source,legacy_data.yaml default
Data,Shuffle train data,True,user.yaml / legacy_data.yaml
Data,Validation shuffle,False,legacy_data.yaml default
Data,Filter overlong prompts,True,user.yaml
Data,Prompt truncation,error,legacy_data.yaml default
Data,enable_thinking,false,script override
Schedule,Total training steps,100,queue/script override
Schedule,Total epochs,30,ppo_trainer/user.yaml default
Schedule,Validation before train,False,queue/script override
Schedule,Save frequency,10,queue/script override
Schedule,Validation frequency,10,queue/script override
Sequence,Max prompt length,2048,queue/script override
Sequence,Max response length,8192,queue/script override
Sequence,Max model length,10240,queue/script override
Sequence,Actor max token length per GPU,10240,queue/script override
Rollout,Rollout engine,vllm,user.yaml
Rollout,Rollout dtype,bfloat16,rollout.yaml default
Rollout,Train rollout n,8,queue/script override
Rollout,Train rollout temperature,1.0,script override
Rollout,Train rollout top_p,1.0,script override
Rollout,Train rollout do_sample,True,rollout.yaml default
Rollout,Calculate rollout log probs,True,baseline_grpo.yaml / script override
Rollout,Max num batched tokens,10240,queue/script override
Rollout,vLLM GPU memory utilization,0.8,queue/script override
Rollout,Tensor model parallel size,2,rollout.yaml default
Rollout,Free cache engine,True,rollout.yaml default
Validation,Validation rollout n,16,queue/script override
Validation,Validation temperature,0.6,queue/script override
Validation,Validation top_p,0.95,queue/script override
Validation,Validation do_sample,True,queue/script override
Optimization,Optimizer,AdamW,fsdp optimizer config
Optimization,Learning rate,1e-6,GRPO method override
Optimization,LR scheduler,constant,W&B config
Optimization,LR warmup steps,10,script override
Optimization,Weight decay,0.01,script override
Optimization,Betas,"(0.9, 0.999)",W&B config
Optimization,Gradient clip,1.0,script override
PPO/GRPO,Policy loss mode,vanilla,method override
PPO/GRPO,Advantage estimator,grpo,baseline_grpo.yaml
PPO/GRPO,Normalize GRPO advantages by std,False,baseline_grpo.yaml / script override
PPO/GRPO,PPO epochs,1,W&B config
PPO/GRPO,PPO mini batch size,8,queue/script override
PPO/GRPO,PPO micro batch size per GPU,1,user.yaml
PPO/GRPO,Clip ratio low,0.2,script override
PPO/GRPO,Clip ratio high,0.28,script override
PPO/GRPO,Gamma,1.0,ppo_trainer.yaml default
PPO/GRPO,Lambda,1.0,ppo_trainer.yaml default
PPO/GRPO,Actor KL loss coef,0.0,method override
PPO/GRPO,Use KL in reward,False,ppo_trainer/user.yaml
Rollout correction,Importance sampling mode,token,script override
Rollout correction,IS threshold,2.0,script override
FSDP/System,Actor strategy,fsdp,dp_actor.yaml
FSDP/System,FSDP dtype,bfloat16,W&B config
FSDP/System,FSDP model dtype,fp32,W&B config
FSDP/System,Use torch compile,True,W&B config
FSDP/System,GPUs per node,8,queue/script override
FSDP/System,Nodes,1,user.yaml
FSDP/System,GPU type,NVIDIA H200,wandb-metadata
Checkpoint/Logging,Checkpoint root,checkpoints/datasets/sciknoweval/physics,script override
Checkpoint/Logging,Latest checkpointed iteration,100,latest_checkpointed_iteration.txt
Checkpoint/Logging,Max actor checkpoints to keep,1,user.yaml
Checkpoint/Logging,Logger,"console, wandb",ppo_trainer.yaml
Checkpoint/Logging,W&B entity,seongryongjung-chung-ang-university,environment
Checkpoint/Logging,W&B project,SDPO-root,user.yaml project_name
Checkpoint/Logging,W&B group,QWEN3-GRPO-generalization,method override
1 section parameter value source
2 Run identity Base model Qwen/Qwen3-4B queue/script override
3 Run identity Dataset Physics / SciKnowEval physics run_qwen3_generalization.sh
4 Run identity Method GRPO run_qwen3_generalization.sh
5 Run identity Config baseline_grpo run_qwen3_generalization.sh
6 Run identity Experiment qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 run_qwen3_generalization.sh
7 Run identity W&B run run-20260702_073403-o8ivivjg wandb
8 Data Train file datasets/sciknoweval/physics/train.parquet script override
9 Data Validation file datasets/sciknoweval/physics/test.parquet script override
10 Data Train batch size 32 queue/script override
11 Data Train max samples 3200 queue/script override
12 Data Prompt key prompt legacy_data.yaml default
13 Data Reward key data_source legacy_data.yaml default
14 Data Shuffle train data True user.yaml / legacy_data.yaml
15 Data Validation shuffle False legacy_data.yaml default
16 Data Filter overlong prompts True user.yaml
17 Data Prompt truncation error legacy_data.yaml default
18 Data enable_thinking false script override
19 Schedule Total training steps 100 queue/script override
20 Schedule Total epochs 30 ppo_trainer/user.yaml default
21 Schedule Validation before train False queue/script override
22 Schedule Save frequency 10 queue/script override
23 Schedule Validation frequency 10 queue/script override
24 Sequence Max prompt length 2048 queue/script override
25 Sequence Max response length 8192 queue/script override
26 Sequence Max model length 10240 queue/script override
27 Sequence Actor max token length per GPU 10240 queue/script override
28 Rollout Rollout engine vllm user.yaml
29 Rollout Rollout dtype bfloat16 rollout.yaml default
30 Rollout Train rollout n 8 queue/script override
31 Rollout Train rollout temperature 1.0 script override
32 Rollout Train rollout top_p 1.0 script override
33 Rollout Train rollout do_sample True rollout.yaml default
34 Rollout Calculate rollout log probs True baseline_grpo.yaml / script override
35 Rollout Max num batched tokens 10240 queue/script override
36 Rollout vLLM GPU memory utilization 0.8 queue/script override
37 Rollout Tensor model parallel size 2 rollout.yaml default
38 Rollout Free cache engine True rollout.yaml default
39 Validation Validation rollout n 16 queue/script override
40 Validation Validation temperature 0.6 queue/script override
41 Validation Validation top_p 0.95 queue/script override
42 Validation Validation do_sample True queue/script override
43 Optimization Optimizer AdamW fsdp optimizer config
44 Optimization Learning rate 1e-6 GRPO method override
45 Optimization LR scheduler constant W&B config
46 Optimization LR warmup steps 10 script override
47 Optimization Weight decay 0.01 script override
48 Optimization Betas (0.9, 0.999) W&B config
49 Optimization Gradient clip 1.0 script override
50 PPO/GRPO Policy loss mode vanilla method override
51 PPO/GRPO Advantage estimator grpo baseline_grpo.yaml
52 PPO/GRPO Normalize GRPO advantages by std False baseline_grpo.yaml / script override
53 PPO/GRPO PPO epochs 1 W&B config
54 PPO/GRPO PPO mini batch size 8 queue/script override
55 PPO/GRPO PPO micro batch size per GPU 1 user.yaml
56 PPO/GRPO Clip ratio low 0.2 script override
57 PPO/GRPO Clip ratio high 0.28 script override
58 PPO/GRPO Gamma 1.0 ppo_trainer.yaml default
59 PPO/GRPO Lambda 1.0 ppo_trainer.yaml default
60 PPO/GRPO Actor KL loss coef 0.0 method override
61 PPO/GRPO Use KL in reward False ppo_trainer/user.yaml
62 Rollout correction Importance sampling mode token script override
63 Rollout correction IS threshold 2.0 script override
64 FSDP/System Actor strategy fsdp dp_actor.yaml
65 FSDP/System FSDP dtype bfloat16 W&B config
66 FSDP/System FSDP model dtype fp32 W&B config
67 FSDP/System Use torch compile True W&B config
68 FSDP/System GPUs per node 8 queue/script override
69 FSDP/System Nodes 1 user.yaml
70 FSDP/System GPU type NVIDIA H200 wandb-metadata
71 Checkpoint/Logging Checkpoint root checkpoints/datasets/sciknoweval/physics script override
72 Checkpoint/Logging Latest checkpointed iteration 100 latest_checkpointed_iteration.txt
73 Checkpoint/Logging Max actor checkpoints to keep 1 user.yaml
74 Checkpoint/Logging Logger console, wandb ppo_trainer.yaml
75 Checkpoint/Logging W&B entity seongryongjung-chung-ang-university environment
76 Checkpoint/Logging W&B project SDPO-root user.yaml project_name
77 Checkpoint/Logging W&B group QWEN3-GRPO-generalization method override

24
results/summary.json Normal file
View File

@@ -0,0 +1,24 @@
{
"repo_id": "SeongryongJung/Qwen3-4B-Physics-GRPO-TR",
"output_dir": "/mnt/mole/SDPO/L2T/hf_upload_tr/Qwen3-4B-Physics-GRPO-TR",
"best_step": 100,
"best_val_mean16": 0.6828125,
"final_step": 100,
"final_val_mean16": 0.6828125,
"train_rows": 90,
"val_rows": 10,
"hf_model_files": [
"added_tokens.json",
"chat_template.jinja",
"config.json",
"generation_config.json",
"merges.txt",
"model.safetensors.index.json",
"special_tokens_map.json",
"tokenizer.json",
"tokenizer_config.json",
"vocab.json",
"model-00001-of-00002.safetensors",
"model-00002-of-00002.safetensors"
]
}

3
results/training_score.png Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6de5124859a1f32d9fed6bbfbb7496e94236aedd46aaaf8bfd5c02c4d6457726
size 120349

1591
results/training_score.svg Normal file
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 43 KiB

91
results/training_scores.csv Normal file
View File

@@ -0,0 +1,91 @@
step,critic_score_mean
1,0.6640625
2,0.578125
3,0.5546875
4,0.625
5,0.7578125
6,0.55859375
7,0.703125
8,0.671875
9,0.64453125
11,0.609375
12,0.59375
13,0.515625
14,0.640625
15,0.6796875
16,0.6953125
17,0.65234375
18,0.40234375
19,0.6875
21,0.671875
22,0.625
23,0.46484375
24,0.7265625
25,0.6640625
26,0.7578125
27,0.71875
28,0.65234375
29,0.6875
31,0.65625
32,0.68359375
33,0.6953125
34,0.65625
35,0.6640625
36,0.56640625
37,0.671875
38,0.61328125
39,0.79296875
41,0.75390625
42,0.5703125
43,0.69140625
44,0.61328125
45,0.625
46,0.7578125
47,0.78125
48,0.69921875
49,0.77734375
51,0.66796875
52,0.65625
53,0.71484375
54,0.73828125
55,0.75390625
56,0.68359375
57,0.59765625
58,0.5859375
59,0.62109375
61,0.68359375
62,0.6328125
63,0.6640625
64,0.66796875
65,0.890625
66,0.7265625
67,0.734375
68,0.7265625
69,0.7265625
71,0.75
72,0.7421875
73,0.61328125
74,0.75390625
75,0.859375
76,0.6875
77,0.703125
78,0.78125
79,0.8046875
81,0.7109375
82,0.63671875
83,0.66796875
84,0.671875
85,0.671875
86,0.6328125
87,0.6875
88,0.66015625
89,0.79296875
91,0.81640625
92,0.69921875
93,0.734375
94,0.7265625
95,0.625
96,0.67578125
97,0.6796875
98,0.70703125
99,0.86328125
1 step critic_score_mean
2 1 0.6640625
3 2 0.578125
4 3 0.5546875
5 4 0.625
6 5 0.7578125
7 6 0.55859375
8 7 0.703125
9 8 0.671875
10 9 0.64453125
11 11 0.609375
12 12 0.59375
13 13 0.515625
14 14 0.640625
15 15 0.6796875
16 16 0.6953125
17 17 0.65234375
18 18 0.40234375
19 19 0.6875
20 21 0.671875
21 22 0.625
22 23 0.46484375
23 24 0.7265625
24 25 0.6640625
25 26 0.7578125
26 27 0.71875
27 28 0.65234375
28 29 0.6875
29 31 0.65625
30 32 0.68359375
31 33 0.6953125
32 34 0.65625
33 35 0.6640625
34 36 0.56640625
35 37 0.671875
36 38 0.61328125
37 39 0.79296875
38 41 0.75390625
39 42 0.5703125
40 43 0.69140625
41 44 0.61328125
42 45 0.625
43 46 0.7578125
44 47 0.78125
45 48 0.69921875
46 49 0.77734375
47 51 0.66796875
48 52 0.65625
49 53 0.71484375
50 54 0.73828125
51 55 0.75390625
52 56 0.68359375
53 57 0.59765625
54 58 0.5859375
55 59 0.62109375
56 61 0.68359375
57 62 0.6328125
58 63 0.6640625
59 64 0.66796875
60 65 0.890625
61 66 0.7265625
62 67 0.734375
63 68 0.7265625
64 69 0.7265625
65 71 0.75
66 72 0.7421875
67 73 0.61328125
68 74 0.75390625
69 75 0.859375
70 76 0.6875
71 77 0.703125
72 78 0.78125
73 79 0.8046875
74 81 0.7109375
75 82 0.63671875
76 83 0.66796875
77 84 0.671875
78 85 0.671875
79 86 0.6328125
80 87 0.6875
81 88 0.66015625
82 89 0.79296875
83 91 0.81640625
84 92 0.69921875
85 93 0.734375
86 94 0.7265625
87 95 0.625
88 96 0.67578125
89 97 0.6796875
90 98 0.70703125
91 99 0.86328125

11
results/validation_mean16.csv Normal file
View File

@@ -0,0 +1,11 @@
step,val_mean16
10,0.59453125
20,0.61328125
30,0.63125
40,0.62421875
50,0.6453125
60,0.6375
70,0.63828125
80,0.6390625
90,0.6625
100,0.6828125
1 step val_mean16
2 10 0.59453125
3 20 0.61328125
4 30 0.63125
5 40 0.62421875
6 50 0.6453125
7 60 0.6375
8 70 0.63828125
9 80 0.6390625
10 90 0.6625
11 100 0.6828125