2.5 KiB
2.5 KiB
| 1 | section | parameter | value | source |
|---|---|---|---|---|
| 2 | Run identity | Base model | Qwen/Qwen3-4B | queue/script override |
| 3 | Run identity | Dataset | Tool-use / tooluse | run_qwen3_generalization.sh |
| 4 | Run identity | Method | RLSD_TR | run_qwen3_generalization.sh |
| 5 | Run identity | Config | rlsd | run_qwen3_generalization.sh |
| 6 | Run identity | Experiment | qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 | run_qwen3_generalization.sh |
| 7 | Run identity | W&B run | run-20260702_154716-jrdzd11b | wandb |
| 8 | Data | Train file | datasets/tooluse/train.parquet | script override |
| 9 | Data | Validation file | datasets/tooluse/test.parquet | script override |
| 10 | Data | Train batch size | 32 | queue/script override |
| 11 | Data | Train max samples | 3200 | queue/script override |
| 12 | Schedule | Total training steps | 100 | queue/script override |
| 13 | Schedule | Validation before train | False | queue/script override |
| 14 | Schedule | Save frequency | 10 | queue/script override |
| 15 | Schedule | Validation frequency | 10 | queue/script override |
| 16 | Sequence | Max prompt length | 2048 | queue/script override |
| 17 | Sequence | Max response length | 8192 | queue/script override |
| 18 | Sequence | Max model length | 10240 | queue/script override |
| 19 | Rollout | Train rollout n | 8 | queue/script override |
| 20 | Rollout | Validation rollout n | 16 | queue/script override |
| 21 | Rollout | vLLM GPU memory utilization | 0.8 | queue/script override |
| 22 | Optimization | Learning rate | 1e-6 | RLSD_TR method override |
| 23 | Optimization | Weight decay | 0.01 | script override |
| 24 | PPO/GRPO | PPO mini batch size | 8 | queue/script override |
| 25 | PPO/GRPO | Normalize GRPO advantages by std | False | baseline_grpo.yaml / script override |
| 26 | Rollout correction | Importance sampling mode | token | script override |
| 27 | Rollout correction | IS threshold | 2.0 | script override |
| 28 | Checkpoint/Logging | Checkpoint root | checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 | script override |
| 29 | Checkpoint/Logging | Latest checkpointed iteration | 100 | latest_checkpointed_iteration.txt |
| 30 | Checkpoint/Logging | External actor archive | checkpoints/datasets/tooluse/qwen3gen-tooluse-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/_actor_archive | preserve_actor_checkpoints.py |
| 31 | Checkpoint/Logging | Logger | console, wandb | ppo_trainer.yaml |
| 32 | RLSD_TR | Policy loss mode | rlsd | method override |
| 33 | RLSD_TR | Teacher regularization | trust-region | method override |
| 34 | RLSD_TR | Trust-region mix / teacher update rate | 0.1 | queue/script override |
| 35 | RLSD_TR | Token reweight lambda | 0.5 | queue/script override |
| 36 | RLSD_TR | Token reweight eps_w | 0.2 | queue/script override |
| 37 | RLSD_TR | Token reweight decay steps | 0 | queue/script override |
| 38 | RLSD_TR | Fused kernels | False | method override |