SeongryongJung/Qwen3-4B-Physics-GRPO-TR
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
8100d939185198410977ef6b4fb7c0449d032b38
Qwen3-4B-Physics-GRPO-TR/artifacts/queue.log

5100 lines
1.7 MiB
Plaintext
Raw Normal View History

初始化项目，由ModelHub XC社区提供模型 Model: SeongryongJung/Qwen3-4B-Physics-GRPO-TR Source: Original Platform
2026-08-05 02:49:18 +08:00
[2026-07-02 06:18:50] Starting L2T batch32 Qwen3-4B GRPO/RLSD_TR dataset-first queue
[2026-07-02 06:18:50] Commit: ee6a667
[2026-07-02 06:18:50] Settings: model=Qwen/Qwen3-4B steps=100 train_batch=32 rlsd_tr_batch=32 rollout=8 mini_batch=8 tr_mix=0.1 max_model_len=10240 vllm=0.8
[2026-07-02 06:18:50] Skipping chemistry grpo; completed by previous queue
[2026-07-02 06:18:50] Starting chemistry rlsd_tr
Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8
Dataset: chemistry
Method: rlsd_tr
Config: rlsd
Model: Qwen/Qwen3-4B
Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet
Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet
Ray tmp: /tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B
2026-07-02 06:18:52,310 INFO scripts.py:1364 -- Did not find any active Ray processes.
Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8
Config: rlsd
Task: datasets/sciknoweval/chemistry
Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240
ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}}
2026-07-02 06:19:01,985 INFO worker.py:2007 -- Started a local Ray instance.
/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
(TaskRunner pid=2026458) TaskRunner hostname: mole-workspace-rw, PID: 2026458
(TaskRunner pid=2026458) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2026458) 'calculate_entropy': False,
(TaskRunner pid=2026458) 'calculate_sum_pi_squared': False,
(TaskRunner pid=2026458) 'checkpoint': {'_target_':
(TaskRunner pid=2026458) 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2026458) 'async_save': False,
(TaskRunner pid=2026458) 'load_contents': ['model',
(TaskRunner pid=2026458) 'optimizer',
(TaskRunner pid=2026458) 'extra'],
(TaskRunner pid=2026458) 'save_contents': ['model',
(TaskRunner pid=2026458) 'optimizer',
(TaskRunner pid=2026458) 'extra']},
(TaskRunner pid=2026458) 'clip_ratio': 0.2,
(TaskRunner pid=2026458) 'clip_ratio_c': 3.0,
(TaskRunner pid=2026458) 'clip_ratio_high': 0.28,
(TaskRunner pid=2026458) 'clip_ratio_low': 0.2,
(TaskRunner pid=2026458) 'data_loader_seed': 42,
(TaskRunner pid=2026458) 'entropy_checkpointing': False,
(TaskRunner pid=2026458) 'entropy_coeff': 0,
(TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2026458) 'freeze_vision_tower': False,
(TaskRunner pid=2026458) 'fsdp_config':
(TaskRunner pid=2026458) {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2026458) 'dtype': 'bfloat16',
(TaskRunner pid=2026458) 'entropy_checkpointing': False,
(TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2026458) 'forward_only': False,
(TaskRunner pid=2026458) 'forward_prefetch': False,
(TaskRunner pid=2026458) 'fsdp_size': -1,
(TaskRunner pid=2026458) 'full_determinism': False,
(TaskRunner pid=2026458) 'model_dtype': 'fp32',
(TaskRunner pid=2026458) 'offload_policy': False,
(TaskRunner pid=2026458) 'optimizer_offload': False,
(TaskRunner pid=2026458) 'param_offload': False,
(TaskRunner pid=2026458) 'reshard_after_forward': True,
(TaskRunner pid=2026458) 'seed': 42,
(TaskRunner pid=2026458) 'strategy': 'fsdp',
(TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2026458) 'use_orig_params': False,
(TaskRunner pid=2026458) 'use_torch_compile': True,
(TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2026458) 'grad_clip': 1.0,
(TaskRunner pid=2026458) 'kl_loss_coef': 0.001,
(TaskRunner pid=2026458) 'kl_loss_type': 'low_var_kl',
(TaskRunner pid=2026458) 'loss_agg_mode': 'token-mean',
(TaskRunner pid=2026458) 'loss_scale_factor': None,
(TaskRunner pid=2026458) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2026458) 'betas': [0.9, 0.999],
(TaskRunner pid=2026458) 'clip_grad': 1.0,
(TaskRunner pid=2026458) 'lr': 1e-06,
(TaskRunner pid=2026458) 'lr_scheduler_type': 'constant',
(TaskRunner pid=2026458) 'lr_warmup_steps': 10,
(TaskRunner pid=2026458) 'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2026458) 'min_lr_ratio': 0.0,
(TaskRunner pid=2026458) 'num_cycles': 0.5,
(TaskRunner pid=2026458) 'optimizer': 'AdamW',
(TaskRunner pid=2026458) 'optimizer_impl': 'torch.optim',
(TaskRunner pid=2026458) 'override_optimizer_config': None,
(TaskRunner pid=2026458) 'total_training_steps': -1,
(TaskRunner pid=2026458) 'warmup_style': None,
(TaskRunner pid=2026458) 'weight_decay': 0.01},
(TaskRunner pid=2026458) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig',
(TaskRunner pid=2026458) 'clip_cov_lb': 1.0,
(TaskRunner pid=2026458) 'clip_cov_ratio': 0.0002,
(TaskRunner pid=2026458) 'clip_cov_ub': 5.0,
(TaskRunner pid=2026458) 'kl_cov_ratio': 0.0002,
(TaskRunner pid=2026458) 'loss_mode': 'rlsd',
(TaskRunner pid=2026458) 'ppo_kl_coef': 0.1},
(TaskRunner pid=2026458) 'ppo_epochs': 1,
(TaskRunner pid=2026458) 'ppo_max_token_len_per_gpu': 10240,
(TaskRunner pid=2026458) 'ppo_micro_batch_size': None,
(TaskRunner pid=2026458) 'ppo_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2026458) 'ppo_mini_batch_size': 8,
(TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2026458) 'all_ranks': False,
(TaskRunner pid=2026458) 'enable': False,
(TaskRunner pid=2026458) 'ranks': [],
(TaskRunner pid=2026458) 'save_path': 'outputs/profile',
(TaskRunner pid=2026458) 'tool': None,
(TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2026458) 'analysis': True,
(TaskRunner pid=2026458) 'contents': [],
(TaskRunner pid=2026458) 'discrete': False,
(TaskRunner pid=2026458) 'level': 'level0'},
(TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2026458) 'discrete': False},
(TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2026458) 'step_end': None,
(TaskRunner pid=2026458) 'step_start': 0},
(TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2026458) 'stack_depth': 32,
(TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2026458) 'rollout_n': 8,
(TaskRunner pid=2026458) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2026458) 'mode': 'disabled',
(TaskRunner pid=2026458) 'record_file': None,
(TaskRunner pid=2026458) 'replay_file': None},
(TaskRunner pid=2026458) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig',
(TaskRunner pid=2026458) 'alpha': 0.5,
(TaskRunner pid=2026458) 'distillation_add_tail': True,
(TaskRunner pid=2026458) 'distillation_topk': 100,
(TaskRunner pid=2026458) 'dont_reprompt_on_self_success': True,
(TaskRunner pid=2026458) 'environment_feedback_only_without_solution': True,
(TaskRunner pid=2026458) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig',
(TaskRunner pid=2026458) 'enable': False,
(TaskRunner pid=2026458) 'loss_weight': 0.0,
(TaskRunner pid=2026458) 'mask': 'all'},
(TaskRunner pid=2026458) 'feedback_template': '\n'
(TaskRunner pid=2026458) 'The '
(TaskRunner pid=2026458) 'following '
(TaskRunner pid=2026458) 'is '
(TaskRunner pid=2026458) 'feedback '
(TaskRunner pid=2026458) 'from '
(TaskRunner pid=2026458) 'your '
(TaskRunner pid=2026458) 'unsuccessful '
(TaskRunner pid=2026458) 'earlier '
(TaskRunner pid=2026458) 'attempt:\n'
(TaskRunner pid=2026458) '\n'
(TaskRunner pid=2026458) '{feedback_raw}',
(TaskRunner pid=2026458) 'full_logit_distillation': True,
(TaskRunner pid=2026458) 'include_environment_feedback': True,
(TaskRunner pid=2026458) 'is_clip': 2,
(TaskRunner pid=2026458) 'max_reprompt_len': 10240,
(TaskRunner pid=2026458) 'remove_thinking_from_demonstration': False,
(TaskRunner pid=2026458) 'reprompt_template': '{prompt}{solution}{feedback}\n'
(TaskRunner pid=2026458) '\n'
(TaskRunner pid=2026458) 'Correctly '
(TaskRunner pid=2026458) 'solve '
(TaskRunner pid=2026458) 'the '
(TaskRunner pid=2026458) 'original '
(TaskRunner pid=2026458) 'question.',
(TaskRunner pid=2026458) 'reprompt_truncation': 'right',
(TaskRunner pid=2026458) 'solution_template': '\n'
(TaskRunner pid=2026458) 'Correct '
(TaskRunner pid=2026458) 'solution:\n'
(TaskRunner pid=2026458) '\n'
(TaskRunner pid=2026458) '{successful_previous_attempt}',
(TaskRunner pid=2026458) 'srpo_dynamic_weighting': False,
(TaskRunner pid=2026458) 'srpo_dynamic_weighting_temperature': 1.0,
(TaskRunner pid=2026458) 'success_reward_threshold': 0.5,
(TaskRunner pid=2026458) 'teacher_regularization': 'trust-region',
(TaskRunner pid=2026458) 'teacher_update_rate': 0.1,
(TaskRunner pid=2026458) 'token_reweight_decay_steps': 0,
(TaskRunner pid=2026458) 'token_reweight_eps_w': 0.2,
(TaskRunner pid=2026458) 'token_reweight_lambda': 0.5},
(TaskRunner pid=2026458) 'shuffle': False,
(TaskRunner pid=2026458) 'strategy': 'fsdp',
(TaskRunner pid=2026458) 'sum_pi_squared_checkpointing': False,
(TaskRunner pid=2026458) 'tau_neg': 1.05,
(TaskRunner pid=2026458) 'tau_pos': 1.0,
(TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2026458) 'use_dynamic_bsz': False,
(TaskRunner pid=2026458) 'use_fused_kernels': False,
(TaskRunner pid=2026458) 'use_kl_loss': False,
(TaskRunner pid=2026458) 'use_prefix_grouper': False,
(TaskRunner pid=2026458) 'use_remove_padding': True,
(TaskRunner pid=2026458) 'use_torch_compile': True},
(TaskRunner pid=2026458) 'hybrid_engine': True,
(TaskRunner pid=2026458) 'model': {'_target_': 'verl.workers.config.HFModelConfig',
(TaskRunner pid=2026458) 'custom_chat_template': None,
(TaskRunner pid=2026458) 'enable_activation_offload': False,
(TaskRunner pid=2026458) 'enable_gradient_checkpointing': True,
(TaskRunner pid=2026458) 'exclude_modules': None,
(TaskRunner pid=2026458) 'external_lib': None,
(TaskRunner pid=2026458) 'fused_kernel_options': {'impl_backend': 'torch'},
(TaskRunner pid=2026458) 'hf_config_path': None,
(TaskRunner pid=2026458) 'lora_adapter_path': None,
(TaskRunner pid=2026458) 'lora_alpha': 16,
(TaskRunner pid=2026458) 'lora_rank': 0,
(TaskRunner pid=2026458) 'override_config': {},
(TaskRunner pid=2026458) 'path': 'Qwen/Qwen3-4B',
(TaskRunner pid=2026458) 'target_modules': 'all-linear',
(TaskRunner pid=2026458) 'tiled_mlp': {'enabled': False,
(TaskRunner pid=2026458) 'num_shards': 4},
(TaskRunner pid=2026458) 'tokenizer_path': None,
(TaskRunner pid=2026458) 'trust_remote_code': True,
(TaskRunner pid=2026458) 'use_fused_kernels': False,
(TaskRunner pid=2026458) 'use_liger': False,
(TaskRunner pid=2026458) 'use_remove_padding': True,
(TaskRunner pid=2026458) 'use_shm': False},
(TaskRunner pid=2026458) 'nccl_timeout': 600,
(TaskRunner pid=2026458) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2026458) 'entropy_checkpointing': False,
(TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2026458) 'dtype': 'bfloat16',
(TaskRunner pid=2026458) 'entropy_checkpointing': False,
(TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2026458) 'forward_only': True,
(TaskRunner pid=2026458) 'forward_prefetch': False,
(TaskRunner pid=2026458) 'fsdp_size': -1,
(TaskRunner pid=2026458) 'full_determinism': False,
(TaskRunner pid=2026458) 'model_dtype': 'fp32',
(TaskRunner pid=2026458) 'offload_policy': False,
(TaskRunner pid=2026458) 'optimizer_offload': False,
(TaskRunner pid=2026458) 'param_offload': False,
(TaskRunner pid=2026458) 'reshard_after_forward': True,
(TaskRunner pid=2026458) 'seed': 42,
(TaskRunner pid=2026458) 'strategy': 'fsdp',
(TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2026458) 'use_orig_params': False,
(TaskRunner pid=2026458) 'use_torch_compile': True,
(TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2026458) 'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2026458) 'log_prob_micro_batch_size': None,
(TaskRunner pid=2026458) 'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2026458) 'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2026458) 'all_ranks': False,
(TaskRunner pid=2026458) 'enable': False,
(TaskRunner pid=2026458) 'ranks': [],
(TaskRunner pid=2026458) 'save_path': 'outputs/profile',
(TaskRunner pid=2026458) 'tool': None,
(TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2026458) 'analysis': True,
(TaskRunner pid=2026458) 'contents': [],
(TaskRunner pid=2026458) 'discrete': False,
(TaskRunner pid=2026458) 'level': 'level0'},
(TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2026458) 'discrete': False},
(TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2026458) 'step_end': None,
(TaskRunner pid=2026458) 'step_start': 0},
(TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2026458) 'stack_depth': 32,
(TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2026458) 'rollout_n': 8,
(TaskRunner pid=2026458) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2026458) 'mode': 'disabled',
(TaskRunner pid=2026458) 'record_file': None,
(TaskRunner pid=2026458) 'replay_file': None},
(TaskRunner pid=2026458) 'strategy': 'fsdp',
(TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2026458) 'use_torch_compile': True},
(TaskRunner pid=2026458) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2026458) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig',
(TaskRunner pid=2026458) 'agent_loop_config_path': None,
(TaskRunner pid=2026458) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig',
(TaskRunner pid=2026458) 'name': None,
(TaskRunner pid=2026458) 'path': None},
(TaskRunner pid=2026458) 'default_agent_loop': 'single_turn_agent',
(TaskRunner pid=2026458) 'num_workers': 8},
(TaskRunner pid=2026458) 'calculate_log_probs': True,
(TaskRunner pid=2026458) 'cudagraph_capture_sizes': None,
(TaskRunner pid=2026458) 'data_parallel_size': 1,
(TaskRunner pid=2026458) 'disable_log_stats': True,
(TaskRunner pid=2026458) 'do_sample': True,
(TaskRunner pid=2026458) 'dtype': 'bfloat16',
(TaskRunner pid=2026458) 'enable_chunked_prefill': True,
(TaskRunner pid=2026458) 'enable_prefix_caching': True,
(TaskRunner pid=2026458) 'enable_rollout_routing_replay': False,
(TaskRunner pid=2026458) 'enforce_eager': False,
(TaskRunner pid=2026458) 'engine_kwargs': {'sglang': {}, 'vllm': {}},
(TaskRunner pid=2026458) 'expert_parallel_size': 1,
(TaskRunner pid=2026458) 'free_cache_engine': True,
(TaskRunner pid=2026458) 'gpu_memory_utilization': 0.8,
(TaskRunner pid=2026458) 'ignore_eos': False,
(TaskRunner pid=2026458) 'layered_summon': False,
(TaskRunner pid=2026458) 'load_format': 'dummy',
(TaskRunner pid=2026458) 'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2026458) 'log_prob_micro_batch_size': None,
(TaskRunner pid=2026458) 'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2026458) 'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2026458) 'logprobs_mode': 'processed_logprobs',
(TaskRunner pid=2026458) 'max_model_len': 10240,
(TaskRunner pid=2026458) 'max_num_batched_tokens': 10240,
(TaskRunner pid=2026458) 'max_num_seqs': 1024,
(TaskRunner pid=2026458) 'mode': 'async',
(TaskRunner pid=2026458) 'multi_stage_wake_up': False,
(TaskRunner pid=2026458) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig',
(TaskRunner pid=2026458) 'enable': False,
(TaskRunner pid=2026458) 'format': 'hermes',
(TaskRunner pid=2026458) 'interaction_config_path': None,
(TaskRunner pid=2026458) 'max_assistant_turns': None,
(TaskRunner pid=2026458) 'max_parallel_calls': 1,
(TaskRunner pid=2026458) 'max_tool_response_length': 256,
(TaskRunner pid=2026458) 'max_user_turns': None,
(TaskRunner pid=2026458) 'num_repeat_rollouts': None,
(TaskRunner pid=2026458) 'tokenization_sanity_check_mode': 'strict',
(TaskRunner pid=2026458) 'tool_config_path': None,
(TaskRunner pid=2026458) 'tool_response_truncate_side': 'middle',
(TaskRunner pid=2026458) 'use_inference_chat_template': False},
(TaskRunner pid=2026458) 'n': 8,
(TaskRunner pid=2026458) 'name': 'vllm',
(TaskRunner pid=2026458) 'over_sample_rate': 0,
(TaskRunner pid=2026458) 'pipeline_model_parallel_size': 1,
(TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2026458) 'all_ranks': False,
(TaskRunner pid=2026458) 'enable': False,
(TaskRunner pid=2026458) 'ranks': [],
(TaskRunner pid=2026458) 'save_path': 'outputs/profile',
(TaskRunner pid=2026458) 'tool': None,
(TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2026458) 'analysis': True,
(TaskRunner pid=2026458) 'contents': [],
(TaskRunner pid=2026458) 'discrete': False,
(TaskRunner pid=2026458) 'level': 'level0'},
(TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2026458) 'discrete': False},
(TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2026458) 'step_end': None,
(TaskRunner pid=2026458) 'step_start': 0},
(TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2026458) 'stack_depth': 32,
(TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2026458) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig',
(TaskRunner pid=2026458) 'enable': False,
(TaskRunner pid=2026458) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml',
(TaskRunner pid=2026458) 'port': 9090,
(TaskRunner pid=2026458) 'served_model_name': 'Qwen/Qwen3-4B'},
(TaskRunner pid=2026458) 'prompt_length': 2048,
(TaskRunner pid=2026458) 'quantization': None,
(TaskRunner pid=2026458) 'quantization_config_file': None,
(TaskRunner pid=2026458) 'response_length': 8192,
(TaskRunner pid=2026458) 'scheduling_policy': 'fcfs',
(TaskRunner pid=2026458) 'skip_dump_dir': '/tmp/rollout_dump',
(TaskRunner pid=2026458) 'skip_rollout': False,
(TaskRunner pid=2026458) 'skip_tokenizer_init': True,
(TaskRunner pid=2026458) 'temperature': 1.0,
(TaskRunner pid=2026458) 'tensor_model_parallel_size': 2,
(TaskRunner pid=2026458) 'top_k': -1,
(TaskRunner pid=2026458) 'top_p': 1.0,
(TaskRunner pid=2026458) 'trace': {'_target_': 'verl.workers.config.TraceConfig',
(TaskRunner pid=2026458) 'backend': None,
(TaskRunner pid=2026458) 'max_samples_per_step_per_worker': None,
(TaskRunner pid=2026458) 'token2text': False},
(TaskRunner pid=2026458) 'update_weights_bucket_megabytes': 512,
(TaskRunner pid=2026458) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig',
(TaskRunner pid=2026458) 'do_sample': True,
(TaskRunner pid=2026458) 'n': 16,
(TaskRunner pid=2026458) 'temperature': 0.6,
(TaskRunner pid=2026458) 'top_k': -1,
(TaskRunner pid=2026458) 'top_p': 0.95}}},
(TaskRunner pid=2026458) 'algorithm':
(TaskRunner pid=2026458) {'_target_': 'verl.trainer.config.AlgoConfig',
(TaskRunner pid=2026458) 'adv_estimator': 'grpo',
(TaskRunner pid=2026458) 'gamma': 1.0,
(TaskRunner pid=2026458) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig',
(TaskRunner pid=2026458) 'horizon': 10000,
(TaskRunner pid=2026458) 'kl_coef': 0.001,
(TaskRunner pid=2026458) 'target_kl': 0.1,
(TaskRunner pid=2026458) 'type': 'fixed'},
(TaskRunner pid=2026458) 'kl_penalty': 'kl',
(TaskRunner pid=2026458) 'lam': 1.0,
(TaskRunner pid=2026458) 'norm_adv_by_std_in_grpo': False,
(TaskRunner pid=2026458) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0},
(TaskRunner pid=2026458) 'rollout_correction': {'bypass_mode': False,
(TaskRunner pid=2026458) 'loss_type': 'ppo_clip',
(TaskRunner pid=2026458) 'rollout_is': 'token',
(TaskRunner pid=2026458) 'rollout_is_batch_normalize': False,
(TaskRunner pid=2026458) 'rollout_is_threshold': 2.0,
(TaskRunner pid=2026458) 'rollout_rs': None,
(TaskRunner pid=2026458) 'rollout_rs_threshold': None},
(TaskRunner pid=2026458) 'use_kl_in_reward': False,
(TaskRunner pid=2026458) 'use_pf_ppo': False},
(TaskRunner pid=2026458) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig',
(TaskRunner pid=2026458) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2026458) 'async_save': False,
(TaskRunner pid=2026458) 'load_contents': ['model', 'optimizer', 'extra'],
(TaskRunner pid=2026458) 'save_contents': ['model', 'optimizer', 'extra']},
(TaskRunner pid=2026458) 'cliprange_value': 0.5,
(TaskRunner pid=2026458) 'data_loader_seed': 42,
(TaskRunner pid=2026458) 'enable': None,
(TaskRunner pid=2026458) 'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2026458) 'forward_micro_batch_size': None,
(TaskRunner pid=2026458) 'forward_micro_batch_size_per_gpu': None,
(TaskRunner pid=2026458) 'grad_clip': 1.0,
(TaskRunner pid=2026458) 'loss_agg_mode': 'token-mean',
(TaskRunner pid=2026458) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg',
(TaskRunner pid=2026458) 'enable_activation_offload': False,
(TaskRunner pid=2026458) 'enable_gradient_checkpointing': True,
(TaskRunner pid=2026458) 'external_lib': None,
(TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2026458) 'dtype': 'bfloat16',
(TaskRunner pid=2026458) 'entropy_checkpointing': False,
(TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2026458) 'forward_only': False,
(TaskRunner pid=2026458) 'forward_prefetch': False,
(TaskRunner pid=2026458) 'fsdp_size': -1,
(TaskRunner pid=2026458) 'full_determinism': False,
(TaskRunner pid=2026458) 'model_dtype': 'fp32',
(TaskRunner pid=2026458) 'offload_policy': False,
(TaskRunner pid=2026458) 'optimizer_offload': False,
(TaskRunner pid=2026458) 'param_offload': False,
(TaskRunner pid=2026458) 'reshard_after_forward': True,
(TaskRunner pid=2026458) 'seed': 42,
(TaskRunner pid=2026458) 'strategy': 'fsdp',
(TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2026458) 'use_orig_params': False,
(TaskRunner pid=2026458) 'use_torch_compile': True,
(TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2026458) 'lora_alpha': 16,
(TaskRunner pid=2026458) 'lora_rank': 0,
(TaskRunner pid=2026458) 'override_config': {},
(TaskRunner pid=2026458) 'path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2026458) 'target_modules': 'all-linear',
(TaskRunner pid=2026458) 'tiled_mlp': {'enabled': False, 'num_shards': 4},
(TaskRunner pid=2026458) 'tokenizer_path': 'Qwen/Qwen3-4B',
(TaskRunner pid=2026458) 'trust_remote_code': True,
(TaskRunner pid=2026458) 'use_remove_padding': False,
(TaskRunner pid=2026458) 'use_shm': False},
(TaskRunner pid=2026458) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2026458) 'betas': [0.9, 0.999],
(TaskRunner pid=2026458) 'clip_grad': 1.0,
(TaskRunner pid=2026458) 'lr': 1e-05,
(TaskRunner pid=2026458) 'lr_scheduler_type': 'constant',
(TaskRunner pid=2026458) 'lr_warmup_steps': -1,
(TaskRunner pid=2026458) 'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2026458) 'min_lr_ratio': 0.0,
(TaskRunner pid=2026458) 'num_cycles': 0.5,
(TaskRunner pid=2026458) 'optimizer': 'AdamW',
(TaskRunner pid=2026458) 'optimizer_impl': 'torch.optim',
(TaskRunner pid=2026458) 'override_optimizer_config': None,
(TaskRunner pid=2026458) 'total_training_steps': -1,
(TaskRunner pid=2026458) 'warmup_style': None,
(TaskRunner pid=2026458) 'weight_decay': 0.01},
(TaskRunner pid=2026458) 'ppo_epochs': 1,
(TaskRunner pid=2026458) 'ppo_max_token_len_per_gpu': 32768,
(TaskRunner pid=2026458) 'ppo_micro_batch_size': None,
(TaskRunner pid=2026458) 'ppo_micro_batch_size_per_gpu': None,
(TaskRunner pid=2026458) 'ppo_mini_batch_size': 8,
(TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2026458) 'all_ranks': False,
(TaskRunner pid=2026458) 'enable': False,
(TaskRunner pid=2026458) 'ranks': [],
(TaskRunner pid=2026458) 'save_path': 'outputs/profile',
(TaskRunner pid=2026458) 'tool': None,
(TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2026458) 'analysis': True,
(TaskRunner pid=2026458) 'contents': [],
(TaskRunner pid=2026458) 'discrete': False,
(TaskRunner pid=2026458) 'level': 'level0'},
(TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2026458) 'discrete': False},
(TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2026458) 'step_end': None,
(TaskRunner pid=2026458) 'step_start': 0},
(TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2026458) 'stack_depth': 32,
(TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2026458) 'rollout_n': 8,
(TaskRunner pid=2026458) 'shuffle': False,
(TaskRunner pid=2026458) 'strategy': 'fsdp',
(TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2026458) 'use_dynamic_bsz': False},
(TaskRunner pid=2026458) 'custom_reward_function': {'name': 'compute_score',
(TaskRunner pid=2026458) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'},
(TaskRunner pid=2026458) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False},
(TaskRunner pid=2026458) 'custom_cls': {'name': None, 'path': None},
(TaskRunner pid=2026458) 'datagen': {'name': None, 'path': None},
(TaskRunner pid=2026458) 'dataloader_num_workers': 8,
(TaskRunner pid=2026458) 'filter_overlong_prompts': True,
(TaskRunner pid=2026458) 'filter_overlong_prompts_workers': 1,
(TaskRunner pid=2026458) 'image_key': 'images',
(TaskRunner pid=2026458) 'image_patch_size': 14,
(TaskRunner pid=2026458) 'max_prompt_length': 2048,
(TaskRunner pid=2026458) 'max_response_length': 8192,
(TaskRunner pid=2026458) 'prompt_key': 'prompt',
(TaskRunner pid=2026458) 'return_full_prompt': False,
(TaskRunner pid=2026458) 'return_multi_modal_inputs': True,
(TaskRunner pid=2026458) 'return_raw_chat': True,
(TaskRunner pid=2026458) 'return_raw_input_ids': False,
(TaskRunner pid=2026458) 'reward_fn_key': 'data_source',
(TaskRunner pid=2026458) 'sampler': {'class_name': None, 'class_path': None},
(TaskRunner pid=2026458) 'seed': None,
(TaskRunner pid=2026458) 'shuffle': True,
(TaskRunner pid=2026458) 'tokenizer': None,
(TaskRunner pid=2026458) 'tool_config_path': None,
(TaskRunner pid=2026458) 'train_batch_size': 32,
(TaskRunner pid=2026458) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'],
(TaskRunner pid=2026458) 'train_max_samples': 3200,
(TaskRunner pid=2026458) 'truncation': 'error',
(TaskRunner pid=2026458) 'trust_remote_code': True,
(TaskRunner pid=2026458) 'use_shm': False,
(TaskRunner pid=2026458) 'val_batch_size': None,
(TaskRunner pid=2026458) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'],
(TaskRunner pid=2026458) 'val_max_samples': -1,
(TaskRunner pid=2026458) 'validation_shuffle': False,
(TaskRunner pid=2026458) 'video_key': 'videos'},
(TaskRunner pid=2026458) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2026458) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2026458) 'controller_nsight_options': {'cuda-graph-trace': 'graph',
(TaskRunner pid=2026458) 'cuda-memory-usage': 'true',
(TaskRunner pid=2026458) 'trace': 'cuda,nvtx,cublas,ucx'},
(TaskRunner pid=2026458) 'discrete': False,
(TaskRunner pid=2026458) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi',
(TaskRunner pid=2026458) 'capture-range-end': None,
(TaskRunner pid=2026458) 'cuda-graph-trace': 'graph',
(TaskRunner pid=2026458) 'cuda-memory-usage': 'true',
(TaskRunner pid=2026458) 'kill': 'none',
(TaskRunner pid=2026458) 'trace': 'cuda,nvtx,cublas,ucx'}},
(TaskRunner pid=2026458) 'torch_memory': {'context': 'all',
(TaskRunner pid=2026458) 'kw_args': {},
(TaskRunner pid=2026458) 'stack_depth': 32,
(TaskRunner pid=2026458) 'stacks': 'all',
(TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}},
(TaskRunner pid=2026458) 'profile_continuous_steps': False,
(TaskRunner pid=2026458) 'save_path': 'outputs/profile',
(TaskRunner pid=2026458) 'steps': None,
(TaskRunner pid=2026458) 'tool': None},
(TaskRunner pid=2026458) 'max_model_len': 10240,
(TaskRunner pid=2026458) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B',
(TaskRunner pid=2026458) 'include_dashboard': False,
(TaskRunner pid=2026458) 'num_cpus': None,
(TaskRunner pid=2026458) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2026458) 'TASK': 'datasets/sciknoweval/chemistry',
(TaskRunner pid=2026458) 'USER': 'root'}}},
(TaskRunner pid=2026458) 'timeline_json_file': None},
(TaskRunner pid=2026458) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig',
(TaskRunner pid=2026458) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig',
(TaskRunner pid=2026458) 'name': 'custom_reward_manager',
(TaskRunner pid=2026458) 'path': None},
(TaskRunner pid=2026458) 'name': 'naive',
(TaskRunner pid=2026458) 'source': 'register'},
(TaskRunner pid=2026458) 'reward_model': {'enable': False,
(TaskRunner pid=2026458) 'enable_resource_pool': False,
(TaskRunner pid=2026458) 'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2026458) 'launch_reward_fn_async': False,
(TaskRunner pid=2026458) 'max_length': None,
(TaskRunner pid=2026458) 'micro_batch_size': None,
(TaskRunner pid=2026458) 'micro_batch_size_per_gpu': None,
(TaskRunner pid=2026458) 'model': {'external_lib': None,
(TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2026458) 'forward_prefetch': False,
(TaskRunner pid=2026458) 'fsdp_size': -1,
(TaskRunner pid=2026458) 'param_offload': False,
(TaskRunner pid=2026458) 'reshard_after_forward': True,
(TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2026458) 'input_tokenizer': 'Qwen/Qwen3-4B',
(TaskRunner pid=2026458) 'override_config': {},
(TaskRunner pid=2026458) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1',
(TaskRunner pid=2026458) 'trust_remote_code': False,
(TaskRunner pid=2026458) 'use_fused_kernels': False,
(TaskRunner pid=2026458) 'use_remove_padding': False,
(TaskRunner pid=2026458) 'use_shm': False},
(TaskRunner pid=2026458) 'n_gpus_per_node': 8,
(TaskRunner pid=2026458) 'nnodes': 0,
(TaskRunner pid=2026458) 'num_workers': 1,
(TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2026458) 'all_ranks': False,
(TaskRunner pid=2026458) 'enable': False,
(TaskRunner pid=2026458) 'ranks': [],
(TaskRunner pid=2026458) 'save_path': 'outputs/profile',
(TaskRunner pid=2026458) 'tool': None,
(TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2026458) 'analysis': True,
(TaskRunner pid=2026458) 'contents': [],
(TaskRunner pid=2026458) 'discrete': False,
(TaskRunner pid=2026458) 'level': 'level0'},
(TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2026458) 'discrete': False},
(TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2026458) 'step_end': None,
(TaskRunner pid=2026458) 'step_start': 0},
(TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2026458) 'stack_depth': 32,
(TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2026458) 'reward_loop_class_name': None,
(TaskRunner pid=2026458) 'reward_loop_module_path': None,
(TaskRunner pid=2026458) 'reward_loop_source': 'register',
(TaskRunner pid=2026458) 'reward_manager': 'naive',
(TaskRunner pid=2026458) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2026458) 'cudagraph_capture_sizes': None,
(TaskRunner pid=2026458) 'data_parallel_size': 1,
(TaskRunner pid=2026458) 'disable_log_stats': True,
(TaskRunner pid=2026458) 'dtype': 'bfloat16',
(TaskRunner pid=2026458) 'enable_chunked_prefill': True,
(TaskRunner pid=2026458) 'enable_prefix_caching': True,
(TaskRunner pid=2026458) 'enforce_eager': True,
(TaskRunner pid=2026458) 'engine_kwargs': {},
(TaskRunner pid=2026458) 'expert_parallel_size': 1,
(TaskRunner pid=2026458) 'free_cache_engine': True,
(TaskRunner pid=2026458) 'gpu_memory_utilization': 0.5,
(TaskRunner pid=2026458) 'limit_images': None,
(TaskRunner pid=2026458) 'load_format': 'auto',
(TaskRunner pid=2026458) 'max_model_len': None,
(TaskRunner pid=2026458) 'max_num_batched_tokens': 8192,
(TaskRunner pid=2026458) 'max_num_seqs': 1024,
(TaskRunner pid=2026458) 'name': '???',
(TaskRunner pid=2026458) 'prompt_length': 2048,
(TaskRunner pid=2026458) 'response_length': 2048,
(TaskRunner pid=2026458) 'skip_tokenizer_init': False,
(TaskRunner pid=2026458) 'tensor_model_parallel_size': 2},
(TaskRunner pid=2026458) 'sandbox_fusion': {'max_concurrent': 64,
(TaskRunner pid=2026458) 'memory_limit_mb': 1024,
(TaskRunner pid=2026458) 'url': None},
(TaskRunner pid=2026458) 'strategy': 'fsdp',
(TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2026458) 'use_dynamic_bsz': False,
(TaskRunner pid=2026458) 'use_reward_loop': False},
(TaskRunner pid=2026458) 'trainer': {'balance_batch': True,
(TaskRunner pid=2026458) 'critic_warmup': 0,
(TaskRunner pid=2026458) 'default_hdfs_dir': None,
(TaskRunner pid=2026458) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2026458) 'del_local_ckpt_after_load': False,
(TaskRunner pid=2026458) 'device': 'cuda',
(TaskRunner pid=2026458) 'esi_redundant_time': 0,
(TaskRunner pid=2026458) 'experiment_name': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2026458) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization',
(TaskRunner pid=2026458) 'log_val_generations': 0,
(TaskRunner pid=2026458) 'logger': ['console', 'wandb'],
(TaskRunner pid=2026458) 'max_actor_ckpt_to_keep': 1,
(TaskRunner pid=2026458) 'max_critic_ckpt_to_keep': None,
(TaskRunner pid=2026458) 'n_gpus_per_node': 8,
(TaskRunner pid=2026458) 'nnodes': 1,
(TaskRunner pid=2026458) 'project_name': 'SDPO-root',
(TaskRunner pid=2026458) 'ray_wait_register_center_timeout': 300,
(TaskRunner pid=2026458) 'resume_from_path': None,
(TaskRunner pid=2026458) 'resume_mode': 'auto',
(TaskRunner pid=2026458) 'rollout_data_dir': None,
(TaskRunner pid=2026458) 'save_freq': 10,
(TaskRunner pid=2026458) 'test_freq': 10,
(TaskRunner pid=2026458) 'total_epochs': 30,
(TaskRunner pid=2026458) 'total_training_steps': 100,
(TaskRunner pid=2026458) 'use_legacy_worker_impl': 'auto',
(TaskRunner pid=2026458) 'val_before_train': False,
(TaskRunner pid=2026458) 'val_only': False,
(TaskRunner pid=2026458) 'validation_data_dir': None},
(TaskRunner pid=2026458) 'transfer_queue': {'enable': False},
(TaskRunner pid=2026458) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry',
(TaskRunner pid=2026458) 'dir': '/users/root/SDPO',
(TaskRunner pid=2026458) 'log_dir': '/users/root/output',
(TaskRunner pid=2026458) 'task': 'datasets/sciknoweval/chemistry'}}
(TaskRunner pid=2026458) [validate_config] All configuration checks passed successfully!
(TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2026458) use_critic=need_critic(config),
(TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2026458) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(TaskRunner pid=2026458) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: eb9682da-997b-4a49-aefd-e3ec8c865f3b)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/tokenizer_config.json
(TaskRunner pid=2026458) WARNING:2026-07-02 06:19:07,289:'(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: eb9682da-997b-4a49-aefd-e3ec8c865f3b)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/tokenizer_config.json
(TaskRunner pid=2026458) Retrying in 1s [Retry 1/5].
(TaskRunner pid=2026458) WARNING:2026-07-02 06:19:07,290:Retrying in 1s [Retry 1/5].
(TaskRunner pid=2026458) Using dataset class: RLHFDataset
(TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(TaskRunner pid=2026458) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(TaskRunner pid=2026458) dataset len: 1890
(TaskRunner pid=2026458) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2026458) WARNING:2026-07-02 06:19:12,124:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/1890 [00:00<?, ? examples/s]
(TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 53%|█████▎ | 1000/1890 [00:00<00:00, 1074.80 examples/s]
(TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 1890/1890 [00:01<00:00, 1558.18 examples/s]
(TaskRunner pid=2026458) filter dataset len: 1890
(TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 1890/1890 [00:01<00:00, 1348.87 examples/s]
(TaskRunner pid=2026458) Using dataset class: RLHFDataset
(TaskRunner pid=2026458) dataset len: 210
(TaskRunner pid=2026458) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2026458) WARNING:2026-07-02 06:19:13,939:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/210 [00:00<?, ? examples/s]
(TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 355.50 examples/s]
(TaskRunner pid=2026458) filter dataset len: 210
(TaskRunner pid=2026458) Size of train dataloader: 59, Size of val dataloader: 1
(TaskRunner pid=2026458) Total training steps: 100
(TaskRunner pid=2026458) colocated worker base class <class 'verl.single_controller.base.worker.Worker'>
(TaskRunner pid=2026458) bind role actor_rollout_ref method chat_completion to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2026458) bind role actor_rollout_ref method generate to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2026458) bind role actor_rollout_ref method get_zeromq_address to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2026458) bind role actor_rollout_ref method sleep to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2026458) bind role actor_rollout_ref method wake_up to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 301.55 examples/s]
(TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2026458) self.use_critic = need_critic(self.config)
(WorkerDict pid=2026872) [W702 06:19:21.430329774 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:52047 (errno: 97 - Address family not supported by protocol).
(WorkerDict pid=2026871) [Gloo] Rank 3 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7
(WorkerDict pid=2026873) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2026873) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2026869) [W702 06:19:22.501122826 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:52047 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(WorkerDict pid=2026868) Model config after override: Qwen3Config {
(WorkerDict pid=2026868) "architectures": [
(WorkerDict pid=2026868) "Qwen3ForCausalLM"
(WorkerDict pid=2026868) ],
(WorkerDict pid=2026868) "attention_bias": false,
(WorkerDict pid=2026868) "attention_dropout": 0.0,
(WorkerDict pid=2026868) "dtype": "bfloat16",
(WorkerDict pid=2026868) "eos_token_id": 151645,
(WorkerDict pid=2026868) "head_dim": 128,
(WorkerDict pid=2026868) "hidden_act": "silu",
(WorkerDict pid=2026868) "hidden_size": 2560,
(WorkerDict pid=2026868) "initializer_range": 0.02,
(WorkerDict pid=2026868) "intermediate_size": 9728,
(WorkerDict pid=2026868) "layer_types": [
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention"
(WorkerDict pid=2026868) ],
(WorkerDict pid=2026868) "max_position_embeddings": 40960,
(WorkerDict pid=2026868) "max_window_layers": 36,
(WorkerDict pid=2026868) "model_type": "qwen3",
(WorkerDict pid=2026868) "num_attention_heads": 32,
(WorkerDict pid=2026868) "num_hidden_layers": 36,
(WorkerDict pid=2026868) "num_key_value_heads": 8,
(WorkerDict pid=2026868) "pad_token_id": 151643,
(WorkerDict pid=2026868) "rms_norm_eps": 1e-06,
(WorkerDict pid=2026868) "rope_scaling": null,
(WorkerDict pid=2026868) "rope_theta": 1000000,
(WorkerDict pid=2026868) "sliding_window": null,
(WorkerDict pid=2026868) "tie_word_embeddings": true,
(WorkerDict pid=2026868) "transformers_version": "4.57.1",
(WorkerDict pid=2026868) "use_cache": true,
(WorkerDict pid=2026868) "use_sliding_window": false,
(WorkerDict pid=2026868) "vocab_size": 151936
(WorkerDict pid=2026868) }
(WorkerDict pid=2026868)
(WorkerDict pid=2026873) `torch_dtype` is deprecated! Use `dtype` instead!
(WorkerDict pid=2026873) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2026873) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2026873) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]
(WorkerDict pid=2026872) Loading checkpoint shards: 33%|███▎ | 1/3 [00:02<00:05, 2.84s/it]
(WorkerDict pid=2026869) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2026869) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2026869) `torch_dtype` is deprecated! Use `dtype` instead! [repeated 7x across cluster]
(WorkerDict pid=2026868) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` [repeated 14x across cluster]
(WorkerDict pid=2026868) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s] [repeated 7x across cluster]
(WorkerDict pid=2026872) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it]
(WorkerDict pid=2026874) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.92s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it]
(WorkerDict pid=2026871) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention
(WorkerDict pid=2026871) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch
(WorkerDict pid=2026869) [Gloo] Rank 1 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 [repeated 7x across cluster]
(WorkerDict pid=2026868) Qwen3ForCausalLM contains 4.02B parameters
(WorkerDict pid=2026868) wrap_policy: functools.partial(<function _or_policy at 0x7473f44a0540>, policies=[functools.partial(<function transformer_auto_wrap_policy at 0x7473f44a0400>, transformer_layer_cls={<class 'transformers.models.qwen3.modeling_qwen3.Qwen3DecoderLayer'>})])
(WorkerDict pid=2026868) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(WorkerDict pid=2026871) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2026870) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.94s/it] [repeated 12x across cluster]
(WorkerDict pid=2026870) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 4x across cluster]
(WorkerDict pid=2026868) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 3.00s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.01s/it] [repeated 2x across cluster]
(WorkerDict pid=2026868) Total steps: 100, num_warmup_steps: 10
(WorkerDict pid=2026868) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster]
(WorkerDict pid=2026868) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster]
(WorkerDict pid=2026868) Actor use_remove_padding=True
(WorkerDict pid=2026868) Actor use_fused_kernels=False
(WorkerDict pid=2026868) Actor use_prefix_grouper=False
(WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2026871) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2026871) [Gloo] Rank 1 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3
(WorkerDict pid=2026871) [Gloo] Rank 1 is connected to 1 peer ranks. Expected number of connected peer ranks is : 1
(WorkerDict pid=2026871) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
(WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
(WorkerDict pid=2026871) warnings.warn(
(WorkerDict pid=2026869) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster]
(WorkerDict pid=2026869) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2026868) reference model: Qwen/Qwen3-4B
(WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster]
(WorkerDict pid=2026871) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster]
(WorkerDict pid=2026868) Model config after override: Qwen3Config {
(WorkerDict pid=2026868) "architectures": [
(WorkerDict pid=2026868) "Qwen3ForCausalLM"
(WorkerDict pid=2026868) ],
(WorkerDict pid=2026868) "attention_bias": false,
(WorkerDict pid=2026868) "attention_dropout": 0.0,
(WorkerDict pid=2026868) "dtype": "bfloat16",
(WorkerDict pid=2026868) "eos_token_id": 151645,
(WorkerDict pid=2026868) "head_dim": 128,
(WorkerDict pid=2026868) "hidden_act": "silu",
(WorkerDict pid=2026868) "hidden_size": 2560,
(WorkerDict pid=2026868) "initializer_range": 0.02,
(WorkerDict pid=2026868) "intermediate_size": 9728,
(WorkerDict pid=2026868) "layer_types": [
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention",
(WorkerDict pid=2026868) "full_attention"
(WorkerDict pid=2026868) ],
(WorkerDict pid=2026868) "max_position_embeddings": 40960,
(WorkerDict pid=2026868) "max_window_layers": 36,
(WorkerDict pid=2026868) "model_type": "qwen3",
(WorkerDict pid=2026868) "num_attention_heads": 32,
(WorkerDict pid=2026868) "num_hidden_layers": 36,
(WorkerDict pid=2026868) "num_key_value_heads": 8,
(WorkerDict pid=2026868) "pad_token_id": 151643,
(WorkerDict pid=2026868) "rms_norm_eps": 1e-06,
(WorkerDict pid=2026868) "rope_scaling": null,
(WorkerDict pid=2026868) "rope_theta": 1000000,
(WorkerDict pid=2026868) "sliding_window": null,
(WorkerDict pid=2026868) "tie_word_embeddings": true,
(WorkerDict pid=2026868) "transformers_version": "4.57.1",
(WorkerDict pid=2026868) "use_cache": true,
(WorkerDict pid=2026868) "use_sliding_window": false,
(WorkerDict pid=2026868) "vocab_size": 151936
(WorkerDict pid=2026868) }
(WorkerDict pid=2026868)
(WorkerDict pid=2026871) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]
(WorkerDict pid=2026871) Loading checkpoint shards: 33%|███▎ | 1/3 [00:02<00:05, 2.87s/it]
(WorkerDict pid=2026875) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster]
(WorkerDict pid=2026875) warnings.warn( [repeated 7x across cluster]
(WorkerDict pid=2026871) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.89s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it]
(WorkerDict pid=2026870) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2026870) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2026874) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s] [repeated 7x across cluster]
(WorkerDict pid=2026868) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it]
(WorkerDict pid=2026871) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention
(WorkerDict pid=2026871) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch
(WorkerDict pid=2026869) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 21x across cluster]
(WorkerDict pid=2026868) Qwen3ForCausalLM contains 4.02B parameters
(WorkerDict pid=2026868) wrap_policy: functools.partial(<function _or_policy at 0x7473f44a0540>, policies=[functools.partial(<function transformer_auto_wrap_policy at 0x7473f44a0400>, transformer_layer_cls={<class 'transformers.models.qwen3.modeling_qwen3.Qwen3DecoderLayer'>})])
(WorkerDict pid=2026868) Ref use_remove_padding=True
(WorkerDict pid=2026868) Ref use_fused_kernels=False
(WorkerDict pid=2026868) Ref use_prefix_grouper=False
(TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2026458) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2026874) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.90s/it] [repeated 12x across cluster]
(WorkerDict pid=2026873) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.89s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 2x across cluster]
(WorkerDict pid=2026874) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 4x across cluster]
(vLLMHttpServer pid=2027750) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(vLLMHttpServer pid=2027750) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(vLLMHttpServer pid=2027750) ['serve',
(vLLMHttpServer pid=2027750) 'Qwen/Qwen3-4B',
(vLLMHttpServer pid=2027750) '--dtype',
(vLLMHttpServer pid=2027750) 'bfloat16',
(vLLMHttpServer pid=2027750) '--load_format',
(vLLMHttpServer pid=2027750) 'dummy',
(vLLMHttpServer pid=2027750) '--trust_remote_code',
(vLLMHttpServer pid=2027750) '--max_model_len',
(vLLMHttpServer pid=2027750) '40960',
(vLLMHttpServer pid=2027750) '--max_num_seqs',
(vLLMHttpServer pid=2027750) '1024',
(vLLMHttpServer pid=2027750) '--enable_chunked_prefill',
(vLLMHttpServer pid=2027750) '--max_num_batched_tokens',
(vLLMHttpServer pid=2027750) '10240',
(vLLMHttpServer pid=2027750) '--enable_prefix_caching',
(vLLMHttpServer pid=2027750) '--enable_sleep_mode',
(vLLMHttpServer pid=2027750) '--logprobs_mode',
(vLLMHttpServer pid=2027750) 'processed_logprobs',
(vLLMHttpServer pid=2027750) '--disable_custom_all_reduce',
(vLLMHttpServer pid=2027750) '--gpu_memory_utilization',
(vLLMHttpServer pid=2027750) '0.8',
(vLLMHttpServer pid=2027750) '--disable_log_stats',
(vLLMHttpServer pid=2027750) '--tensor_parallel_size',
(vLLMHttpServer pid=2027750) '2',
(vLLMHttpServer pid=2027750) '--seed',
(vLLMHttpServer pid=2027750) '0',
(vLLMHttpServer pid=2027750) '--override_generation_config',
(vLLMHttpServer pid=2027750) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, '
(vLLMHttpServer pid=2027750) '"max_new_tokens": 8192}',
(vLLMHttpServer pid=2027750) '--hf_overrides',
(vLLMHttpServer pid=2027750) '{}',
(vLLMHttpServer pid=2027750) '--scheduling_policy',
(vLLMHttpServer pid=2027750) 'fcfs']
(WorkerDict pid=2026872) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster]
(WorkerDict pid=2026872) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster]
(vLLMHttpServer pid=2027750) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead.
(vLLMHttpServer pid=2027750) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(vLLMHttpServer pid=2027750) WARNING 07-02 06:20:18 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned
(WorkerDict pid=2026868) WARNING 07-02 06:20:26 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'.
(vLLMHttpServer pid=2027751) WARNING 07-02 06:20:19 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster]
(WorkerDict pid=2026872) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2026868) 2026-07-02 06:20:40,724 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ...
(WorkerDict pid=2026868) 2026-07-02 06:20:40,743 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends
(vLLMHttpServer pid=2027753) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster]
(vLLMHttpServer pid=2027753) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster]
(vLLMHttpServer pid=2027753) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster]
(vLLMHttpServer pid=2027753) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 4%|▍ | 2/51 [00:00<00:02, 17.73it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 8%|▊ | 4/51 [00:00<00:02, 17.68it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 12%|█▏ | 6/51 [00:00<00:02, 17.21it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 16%|█▌ | 8/51 [00:00<00:02, 17.00it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 20%|█▉ | 10/51 [00:00<00:02, 17.41it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 24%|██▎ | 12/51 [00:00<00:02, 17.57it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 27%|██▋ | 14/51 [00:00<00:02, 17.05it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 31%|███▏ | 16/51 [00:00<00:02, 17.01it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 35%|███▌ | 18/51 [00:01<00:01, 16.54it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 39%|███▉ | 20/51 [00:01<00:01, 16.58it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 43%|████▎ | 22/51 [00:01<00:01, 16.19it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 47%|████▋ | 24/51 [00:01<00:01, 15.98it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 51%|█████ | 26/51 [00:01<00:01, 16.06it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 55%|█████▍ | 28/51 [00:01<00:01, 15.89it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 59%|█████▉ | 30/51 [00:01<00:01, 15.68it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 63%|██████▎ | 32/51 [00:01<00:01, 15.51it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 67%|██████▋ | 34/51 [00:02<00:01, 14.72it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 71%|███████ | 36/51 [00:02<00:01, 14.69it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 75%|███████▍ | 38/51 [00:02<00:00, 14.15it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 78%|███████▊ | 40/51 [00:02<00:00, 13.94it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 82%|████████▏ | 42/51 [00:02<00:00, 14.29it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 86%|████████▋ | 44/51 [00:02<00:00, 14.35it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 90%|█████████ | 46/51 [00:02<00:00, 14.34it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 94%|█████████▍| 48/51 [00:03<00:00, 14.47it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 98%|█████████▊| 50/51 [00:03<00:00, 13.83it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 15.33it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 0%| | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 4%|▍ | 2/51 [00:00<00:02, 16.45it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 10%|▉ | 5/51 [00:00<00:02, 16.78it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 16%|█▌ | 8/51 [00:00<00:02, 19.55it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 22%|██▏ | 11/51 [00:00<00:01, 21.00it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 27%|██▋ | 14/51 [00:00<00:01, 21.41it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 33%|███▎ | 17/51 [00:00<00:01, 22.20it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 39%|███▉ | 20/51 [00:00<00:01, 22.58it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 45%|████▌ | 23/51 [00:01<00:01, 22.91it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 51%|█████ | 26/51 [00:01<00:01, 22.99it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 57%|█████▋ | 29/51 [00:01<00:00, 22.82it/s]
(WorkerDict pid=2026875) 2026-07-02 06:20:41,260 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... [repeated 7x across cluster]
(WorkerDict pid=2026875) 2026-07-02 06:20:41,276 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends [repeated 7x across cluster]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 63%|██████▎ | 32/51 [00:01<00:00, 22.28it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 69%|██████▊ | 35/51 [00:01<00:00, 22.31it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 75%|███████▍ | 38/51 [00:01<00:00, 22.57it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 80%|████████ | 41/51 [00:01<00:00, 22.49it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 86%|████████▋ | 44/51 [00:02<00:00, 22.59it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 92%|█████████▏| 47/51 [00:02<00:00, 22.72it/s]
(WorkerDict pid=2026868) Capturing CUDA graphs (decode, FULL): 98%|█████████▊| 50/51 [00:02<00:00, 22.86it/s] Capturing CUDA graphs (decode, FULL): 100%|██████████| 51/51 [00:02<00:00, 22.10it/s]
(vLLMHttpServer pid=2027750) WARNING 07-02 06:20:49 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development!
(WorkerDict pid=2026870) WARNING 07-02 06:20:27 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. [repeated 7x across cluster]
(WorkerDict pid=2026870) NCCL version 2.27.5+cuda12.9 [repeated 2x across cluster]
(vLLMHttpServer pid=2027750) WARNING 07-02 06:20:49 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(TaskRunner pid=2026458) AgentLoopManager: ['198.19.44.212:34373', '198.19.44.212:37023', '198.19.44.212:38415', '198.19.44.212:44581']
(TaskRunner pid=2026458) wandb: Currently logged in as: seongryongjung (seongryongjung-chung-ang-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
(TaskRunner pid=2026458) wandb: setting up run bz6p2yxy
(TaskRunner pid=2026458) wandb: Tracking run with wandb version 0.23.1
(TaskRunner pid=2026458) wandb: Run data is saved locally in /mnt/mole/SDPO/L2T/wandb/run-20260702_062054-bz6p2yxy
(TaskRunner pid=2026458) wandb: Run `wandb offline` to turn off syncing.
(TaskRunner pid=2026458) wandb: Syncing run qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8
(TaskRunner pid=2026458) wandb: ⭐️ View project at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root
(TaskRunner pid=2026458) wandb: 🚀 View run at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/bz6p2yxy
(TaskRunner pid=2026458) Checkpoint tracker file does not exist: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/latest_checkpointed_iteration.txt
(TaskRunner pid=2026458) Training from scratch
(vLLMHttpServer pid=2027753) WARNING 07-02 06:20:49 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development! [repeated 3x across cluster]
(vLLMHttpServer pid=2027753) WARNING 07-02 06:20:49 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 3x across cluster]
(TaskRunner pid=2026458) wandb: Detected [openai] in use.
(TaskRunner pid=2026458) wandb: Use W&B Weave for improved LLM call tracing. Install Weave with `pip install weave` then add `import weave` to the top of your script.
(TaskRunner pid=2026458) wandb: For more information, check out the docs at: https://weave-docs.wandb.ai/
(TaskRunner pid=2026458) Training Progress: 0%| | 0/100 [00:00<?, ?it/s]
(AgentLoopWorker pid=2028722) Using dataset class: RLHFDataset
(AgentLoopWorker pid=2028722) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(AgentLoopWorker pid=2028722) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(AgentLoopWorker pid=2028730) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
(AgentLoopWorker pid=2028725) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(AgentLoopWorker pid=2028725) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(TaskRunner pid=2026458) step:1 - global_seqlen/min:15161 - global_seqlen/max:20443 - global_seqlen/minmax_diff:5282 - global_seqlen/balanced_min:18656 - global_seqlen/balanced_max:18662 - global_seqlen/mean:18660.5 - actor/entropy:0.227603480219841 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292527914047241 - training/rollout_probs_diff_mean:0.006219237111508846 - training/rollout_probs_diff_std:0.016583416610956192 - training/rollout_actor_probs_pearson_corr:0.9972637891769409 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.5546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:1.000045657157898 - rollout_corr/rollout_is_ratio_fraction_high:0.00010812628897838295 - rollout_corr/rollout_is_ratio_fraction_low:0.00013215435319580138 - rollout_corr/rollout_is_max:7.675390720367432 - rollout_corr/rollout_is_min:0.19046327471733093 - rollout_corr/rollout_is_std:0.047792982310056686 - rollout_corr/rollout_is_eff_sample_size:0.9977207990084411 - rollout_corr/rollout_is_seq_mean:1.000111699104309 - rollout_corr/rollout_is_seq_std:0.0032105317804962397 - rollout_corr/rollout_is_seq_max:1.0260839462280273 - rollout_corr/rollout_is_seq_min:0.9926905632019043 - rollout_corr/rollout_is_seq_max_deviation:0.026083946228027344 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.269956030882895) - rollout_corr/training_log_ppl:np.float64(0.23531292796542402) - rollout_corr/kl:np.float64(0.001176612532361787) - rollout_corr/k3_kl:np.float64(0.0012883886294048352) - rollout_corr/rollout_ppl:np.float64(1.2684132866561413) - rollout_corr/rollout_log_ppl:np.float64(0.23413631394214462) - rollout_corr/log_ppl_diff:np.float64(0.0011766140232793987) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023820645874366164) - rollout_corr/log_ppl_diff_max:np.float64(0.009120479226112366) - rollout_corr/log_ppl_diff_min:np.float64(-0.006215203553438187) - rollout_corr/ppl_ratio:np.float64(1.0011813363526016) - rollout_corr/chi2_token:np.float64(0.0033419926185160875) - rollout_corr/chi2_seq:np.float64(0.7175239583011717) - actor/pg_loss:np.float64(0.0017848979914560914) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5546875) - self_distillation/token_reweight_w_mean:np.float64(149848.50953566656) - self_distillation/token_reweight_w_std:np.float64(2183327.3896330735) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.003698990214616) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11359061126131564) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.7533117979764938) - perf/mfu/actor:np.float64(0.027145070230281433) - perf/max_memory_allocated_gb:np.float64(116.94684362411499) - perf/max_memory_reserved_gb:np.float64(121.140625) - perf/cpu_memory_used_gb:np.float64(154.0656394958496) - actor/lr:np.float64(0.0) - training/global_step:1 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038459920324385166 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0038459920324385166 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.140625 - respon
(AgentLoopWorker pid=2028725) Using dataset class: RLHFDataset [repeated 7x across cluster]
(TaskRunner pid=2026458) Training Progress: 1%| | 1/100 [00:58<1:35:54, 58.12s/it]
(AgentLoopWorker pid=2028728) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding. [repeated 7x across cluster]
(TaskRunner pid=2026458) step:2 - global_seqlen/min:17405 - global_seqlen/max:27636 - global_seqlen/minmax_diff:10231 - global_seqlen/balanced_min:20974 - global_seqlen/balanced_max:23605 - global_seqlen/mean:21303.875 - actor/entropy:0.2105097621679306 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.621828019618988 - training/rollout_probs_diff_mean:0.005872448440641165 - training/rollout_probs_diff_std:0.016020672395825386 - training/rollout_actor_probs_pearson_corr:0.9972701668739319 - self_distillation/success_group_fraction:0.5625 - self_distillation/success_sample_fraction:0.55859375 - self_distillation/correct_sample_fraction:0.3828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.55859375 - rollout_corr/rollout_is_mean:0.9998953938484192 - rollout_corr/rollout_is_ratio_fraction_high:5.6403488997602835e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00019177186186425388 - rollout_corr/rollout_is_max:2.5879361629486084 - rollout_corr/rollout_is_min:0.23851390182971954 - rollout_corr/rollout_is_std:0.047345660626888275 - rollout_corr/rollout_is_eff_sample_size:0.99776304607302 - rollout_corr/rollout_is_seq_mean:0.9999005198478699 - rollout_corr/rollout_is_seq_std:0.002932196483016014 - rollout_corr/rollout_is_seq_max:1.0083532333374023 - rollout_corr/rollout_is_seq_min:0.9910140633583069 - rollout_corr/rollout_is_seq_max_deviation:0.008985936641693115 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2692418885417283) - rollout_corr/training_log_ppl:np.float64(0.235447489049875) - rollout_corr/kl:np.float64(0.0012569587911386293) - rollout_corr/k3_kl:np.float64(0.001248163497166388) - rollout_corr/rollout_ppl:np.float64(1.2676085638813674) - rollout_corr/rollout_log_ppl:np.float64(0.2341905289249553) - rollout_corr/log_ppl_diff:np.float64(0.0012569601249197149) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025652070480646216) - rollout_corr/log_ppl_diff_max:np.float64(0.01221117377281189) - rollout_corr/log_ppl_diff_min:np.float64(-0.005978986620903015) - rollout_corr/ppl_ratio:np.float64(1.0012623998336494) - rollout_corr/chi2_token:np.float64(0.0024625081568956375) - rollout_corr/chi2_seq:np.float64(2.013864086009562) - actor/pg_loss:np.float64(0.006197612965479493) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008408953899561311) - actor/ppo_kl:np.float64(-0.00012492353172177673) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.55859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.3828125) - self_distillation/token_reweight_w_mean:np.float64(157759.15380330454) - self_distillation/token_reweight_w_std:np.float64(2217101.6862332174) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012996185105294) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07309902091037657) - self_distillation/empty_target_batch:np.float64(0.44140625) - actor/grad_norm:np.float64(0.5681237205862999) - perf/mfu/actor:np.float64(0.036623311323520286) - perf/max_memory_allocated_gb:np.float64(118.78376626968384) - perf/max_memory_reserved_gb:np.float64(124.83203125) - perf/cpu_memory_used_gb:np.float64(149.6806983947754) - actor/lr:np.float64(1e-07) - training/global_step:2 - training/epoch:0 - critic/score/mean:0.3828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.3828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.025488736107945442 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.025488736107945442 - critic/returns/max:0.875 - critic/returns/min:-0.75 - respon
(TaskRunner pid=2026458) Training Progress: 2%|▏ | 2/100 [01:45<1:24:44, 51.88s/it]
(TaskRunner pid=2026458) step:3 - global_seqlen/min:17465 - global_seqlen/max:24761 - global_seqlen/minmax_diff:7296 - global_seqlen/balanced_min:20435 - global_seqlen/balanced_max:20438 - global_seqlen/mean:20436.5 - actor/entropy:0.23337818682193756 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5623742341995239 - training/rollout_probs_diff_mean:0.006082512903958559 - training/rollout_probs_diff_std:0.01588571071624756 - training/rollout_actor_probs_pearson_corr:0.9975335001945496 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.000319004058838 - rollout_corr/rollout_is_ratio_fraction_high:0.00010457820462761447 - rollout_corr/rollout_is_ratio_fraction_low:0.00015105740749277174 - rollout_corr/rollout_is_max:2.8437411785125732 - rollout_corr/rollout_is_min:0.19363267719745636 - rollout_corr/rollout_is_std:0.04873770847916603 - rollout_corr/rollout_is_eff_sample_size:0.9976315697774746 - rollout_corr/rollout_is_seq_mean:1.0003259181976318 - rollout_corr/rollout_is_seq_std:0.002991831162944436 - rollout_corr/rollout_is_seq_max:1.0120688676834106 - rollout_corr/rollout_is_seq_min:0.9895726442337036 - rollout_corr/rollout_is_seq_max_deviation:0.012068867683410645 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2909718337468803) - rollout_corr/training_log_ppl:np.float64(0.2515612288698321) - rollout_corr/kl:np.float64(0.0010431841244908924) - rollout_corr/k3_kl:np.float64(0.00129854386665329) - rollout_corr/rollout_ppl:np.float64(1.289595266804099) - rollout_corr/rollout_log_ppl:np.float64(0.2505180448642932) - rollout_corr/log_ppl_diff:np.float64(0.001043184005538933) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023550822370452806) - rollout_corr/log_ppl_diff_max:np.float64(0.015129491686820984) - rollout_corr/log_ppl_diff_min:np.float64(-0.009931474924087524) - rollout_corr/ppl_ratio:np.float64(1.0010482762008905) - rollout_corr/chi2_token:np.float64(0.0032234792597591877) - rollout_corr/chi2_seq:np.float64(1.1858129473403096) - actor/pg_loss:np.float64(0.010745072388090193) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001127584007463156) - actor/ppo_kl:np.float64(7.391470547268852e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.453125) - self_distillation/token_reweight_w_mean:np.float64(153395.63152351254) - self_distillation/token_reweight_w_std:np.float64(2449559.4464063025) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0016049204859883) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13649266603169963) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.7008706703782082) - perf/mfu/actor:np.float64(0.03742893495516526) - perf/max_memory_allocated_gb:np.float64(118.78376626968384) - perf/max_memory_reserved_gb:np.float64(124.83203125) - perf/cpu_memory_used_gb:np.float64(147.21394157409668) - actor/lr:np.float64(2e-07) - training/global_step:3 - training/epoch:0 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008508598431944847 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008508598431944847 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean
(TaskRunner pid=2026458) Training Progress: 3%|▎ | 3/100 [02:16<1:08:35, 42.43s/it]
(TaskRunner pid=2026458) step:4 - global_seqlen/min:15709 - global_seqlen/max:22726 - global_seqlen/minmax_diff:7017 - global_seqlen/balanced_min:19482 - global_seqlen/balanced_max:20000 - global_seqlen/mean:19547.0 - actor/entropy:0.24194160103797913 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311299562454224 - training/rollout_probs_diff_mean:0.006227096542716026 - training/rollout_probs_diff_std:0.015620573423802853 - training/rollout_actor_probs_pearson_corr:0.9976333379745483 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.63671875 - self_distillation/correct_sample_fraction:0.33984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.63671875 - rollout_corr/rollout_is_mean:1.000150442123413 - rollout_corr/rollout_is_ratio_fraction_high:3.853267480735667e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012844224693253636 - rollout_corr/rollout_is_max:3.7063121795654297 - rollout_corr/rollout_is_min:0.3662450611591339 - rollout_corr/rollout_is_std:0.04812481626868248 - rollout_corr/rollout_is_eff_sample_size:0.9976899466712242 - rollout_corr/rollout_is_seq_mean:1.0001506805419922 - rollout_corr/rollout_is_seq_std:0.003076401771977544 - rollout_corr/rollout_is_seq_max:1.0096232891082764 - rollout_corr/rollout_is_seq_min:0.9903525710105896 - rollout_corr/rollout_is_seq_max_deviation:0.0096474289894104 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2937282132916152) - rollout_corr/training_log_ppl:np.float64(0.25533604525844567) - rollout_corr/kl:np.float64(0.0010333105294506595) - rollout_corr/k3_kl:np.float64(0.0012225817687294693) - rollout_corr/rollout_ppl:np.float64(1.2923662513494492) - rollout_corr/rollout_log_ppl:np.float64(0.2543027339124819) - rollout_corr/log_ppl_diff:np.float64(0.0010333113459637389) - rollout_corr/log_ppl_abs_diff:np.float64(0.002593920609797351) - rollout_corr/log_ppl_diff_max:np.float64(0.013403445482254028) - rollout_corr/log_ppl_diff_min:np.float64(-0.009037211537361145) - rollout_corr/ppl_ratio:np.float64(1.0010389203671366) - rollout_corr/chi2_token:np.float64(0.0028364709578454494) - rollout_corr/chi2_seq:np.float64(0.9269450127612799) - actor/pg_loss:np.float64(0.009128515957854688) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001327361899711832) - actor/ppo_kl:np.float64(-4.086699547012351e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.63671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.33984375) - self_distillation/token_reweight_w_mean:np.float64(85569.56621437636) - self_distillation/token_reweight_w_std:np.float64(1335522.5642696854) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015453973319381) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12542373192263767) - self_distillation/empty_target_batch:np.float64(0.36328125) - actor/grad_norm:np.float64(0.7053019404411316) - perf/mfu/actor:np.float64(0.03616655050820109) - perf/max_memory_allocated_gb:np.float64(118.78376626968384) - perf/max_memory_reserved_gb:np.float64(124.83203125) - perf/cpu_memory_used_gb:np.float64(145.6499366760254) - actor/lr:np.float64(3e-07) - training/global_step:4 - training/epoch:0 - critic/score/mean:0.33984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.33984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004479423630982637 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004479423630982637 - critic/returns/max:0.875 - critic/returns/min:-0.875 - re
(TaskRunner pid=2026458) Training Progress: 4%|▍ | 4/100 [02:49<1:01:51, 38.66s/it]
(TaskRunner pid=2026458) step:5 - global_seqlen/min:15330 - global_seqlen/max:22691 - global_seqlen/minmax_diff:7361 - global_seqlen/balanced_min:18972 - global_seqlen/balanced_max:18979 - global_seqlen/mean:18977.125 - actor/entropy:0.22716574370861053 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535163104534149 - training/rollout_probs_diff_mean:0.005988361779600382 - training/rollout_probs_diff_std:0.015364344231784344 - training/rollout_actor_probs_pearson_corr:0.9976463913917542 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:0.9999639987945557 - rollout_corr/rollout_is_ratio_fraction_high:3.669231227831915e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.33846245566383e-05 - rollout_corr/rollout_is_max:2.3193719387054443 - rollout_corr/rollout_is_min:0.2404157668352127 - rollout_corr/rollout_is_std:0.046586740761995316 - rollout_corr/rollout_is_eff_sample_size:0.9978341383943695 - rollout_corr/rollout_is_seq_mean:0.9999549984931946 - rollout_corr/rollout_is_seq_std:0.002704878803342581 - rollout_corr/rollout_is_seq_max:1.0085281133651733 - rollout_corr/rollout_is_seq_min:0.9920772314071655 - rollout_corr/rollout_is_seq_max_deviation:0.00852811336517334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2816479047760367) - rollout_corr/training_log_ppl:np.float64(0.24531584553187713) - rollout_corr/kl:np.float64(0.0013180274357958432) - rollout_corr/k3_kl:np.float64(0.0011023088188721886) - rollout_corr/rollout_ppl:np.float64(1.2799232499673963) - rollout_corr/rollout_log_ppl:np.float64(0.243997815734474) - rollout_corr/log_ppl_diff:np.float64(0.0013180297974031419) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024417925451416522) - rollout_corr/log_ppl_diff_max:np.float64(0.010013550519943237) - rollout_corr/log_ppl_diff_min:np.float64(-0.007263094186782837) - rollout_corr/ppl_ratio:np.float64(1.001323260134086) - rollout_corr/chi2_token:np.float64(0.0017700856551527977) - rollout_corr/chi2_seq:np.float64(0.5527992681600153) - actor/pg_loss:np.float64(0.00845826999284327) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013763844040113327) - actor/ppo_kl:np.float64(0.0001424315673652643) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.484375) - self_distillation/token_reweight_w_mean:np.float64(243812.5012207271) - self_distillation/token_reweight_w_std:np.float64(3343822.672462601) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.004243799019605) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14025208985549398) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.7075520902872086) - perf/mfu/actor:np.float64(0.03489651122749955) - perf/max_memory_allocated_gb:np.float64(118.78376626968384) - perf/max_memory_reserved_gb:np.float64(124.83203125) - perf/cpu_memory_used_gb:np.float64(144.03763961791992) - actor/lr:np.float64(4e-07) - training/global_step:5 - training/epoch:0 - critic/score/mean:0.484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0054564522579312325 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0054564522579312325 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean
(TaskRunner pid=2026458) Training Progress: 5%|▌ | 5/100 [03:20<57:00, 36.00s/it]
(TaskRunner pid=2026458) step:6 - global_seqlen/min:17784 - global_seqlen/max:21568 - global_seqlen/minmax_diff:3784 - global_seqlen/balanced_min:19891 - global_seqlen/balanced_max:19899 - global_seqlen/mean:19895.5 - actor/entropy:0.2139550894498825 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6129132509231567 - training/rollout_probs_diff_mean:0.005732228513807058 - training/rollout_probs_diff_std:0.01550403330475092 - training/rollout_actor_probs_pearson_corr:0.9974612593650818 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.44140625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:1.0000214576721191 - rollout_corr/rollout_is_ratio_fraction_high:4.504301614360884e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015765056014060974 - rollout_corr/rollout_is_max:6.671880722045898 - rollout_corr/rollout_is_min:0.19801130890846252 - rollout_corr/rollout_is_std:0.04693472385406494 - rollout_corr/rollout_is_eff_sample_size:0.9978018548158412 - rollout_corr/rollout_is_seq_mean:1.0001208782196045 - rollout_corr/rollout_is_seq_std:0.00283028744161129 - rollout_corr/rollout_is_seq_max:1.0104447603225708 - rollout_corr/rollout_is_seq_min:0.9932514429092407 - rollout_corr/rollout_is_seq_max_deviation:0.0104447603225708 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2592799328267574) - rollout_corr/training_log_ppl:np.float64(0.22757626220118254) - rollout_corr/kl:np.float64(0.0008963848200842506) - rollout_corr/k3_kl:np.float64(0.0011540942741703475) - rollout_corr/rollout_ppl:np.float64(1.2581414724700153) - rollout_corr/rollout_log_ppl:np.float64(0.22667987761087716) - rollout_corr/log_ppl_diff:np.float64(0.000896384590305388) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021794571657665074) - rollout_corr/log_ppl_diff_max:np.float64(0.007545456290245056) - rollout_corr/log_ppl_diff_min:np.float64(-0.007387563586235046) - rollout_corr/ppl_ratio:np.float64(1.0009002275764942) - rollout_corr/chi2_token:np.float64(0.0028780833818018436) - rollout_corr/chi2_seq:np.float64(1.2613209481351078) - actor/pg_loss:np.float64(0.00913890392985195) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010952619722957024) - actor/ppo_kl:np.float64(-0.00021712644839455564) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.44140625) - self_distillation/token_reweight_w_mean:np.float64(225756.38810028392) - self_distillation/token_reweight_w_std:np.float64(3639913.945713965) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0023127323947847) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13595641919528134) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.6455599144101143) - perf/mfu/actor:np.float64(0.03683066344058797) - perf/max_memory_allocated_gb:np.float64(118.78376626968384) - perf/max_memory_reserved_gb:np.float64(124.83203125) - perf/cpu_memory_used_gb:np.float64(142.44257736206055) - actor/lr:np.float64(5e-07) - training/global_step:6 - training/epoch:0 - critic/score/mean:0.44140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.44140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0017707535298541188 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0017707535298541188 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_
(TaskRunner pid=2026458) Training Progress: 6%|▌ | 6/100 [03:52<54:11, 34.60s/it]
(TaskRunner pid=2026458) step:7 - global_seqlen/min:16271 - global_seqlen/max:21855 - global_seqlen/minmax_diff:5584 - global_seqlen/balanced_min:18915 - global_seqlen/balanced_max:18918 - global_seqlen/mean:18917.125 - actor/entropy:0.2508375942707062 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553977966308594 - training/rollout_probs_diff_mean:0.006225189194083214 - training/rollout_probs_diff_std:0.01496341172605753 - training/rollout_actor_probs_pearson_corr:0.9978689551353455 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.40625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.9999915361404419 - rollout_corr/rollout_is_ratio_fraction_high:2.585549373179674e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.171098746359348e-05 - rollout_corr/rollout_is_max:4.0770087242126465 - rollout_corr/rollout_is_min:0.40586209297180176 - rollout_corr/rollout_is_std:0.04609152674674988 - rollout_corr/rollout_is_eff_sample_size:0.9978798374720497 - rollout_corr/rollout_is_seq_mean:1.0000202655792236 - rollout_corr/rollout_is_seq_std:0.002876279642805457 - rollout_corr/rollout_is_seq_max:1.0085850954055786 - rollout_corr/rollout_is_seq_min:0.9919223189353943 - rollout_corr/rollout_is_seq_max_deviation:0.008585095405578613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3062554383650422) - rollout_corr/training_log_ppl:np.float64(0.26436210473184474) - rollout_corr/kl:np.float64(0.0010862308800456688) - rollout_corr/k3_kl:np.float64(0.0012081595617701169) - rollout_corr/rollout_ppl:np.float64(1.3048601374030113) - rollout_corr/rollout_log_ppl:np.float64(0.26327587306150235) - rollout_corr/log_ppl_diff:np.float64(0.0010862316703423858) - rollout_corr/log_ppl_abs_diff:np.float64(0.00240726163610816) - rollout_corr/log_ppl_diff_max:np.float64(0.010948970913887024) - rollout_corr/log_ppl_diff_min:np.float64(-0.008498936891555786) - rollout_corr/ppl_ratio:np.float64(1.0010909896809608) - rollout_corr/chi2_token:np.float64(0.002930266549810767) - rollout_corr/chi2_seq:np.float64(0.7651277596596628) - actor/pg_loss:np.float64(0.009830480092205107) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010996697033078817) - actor/ppo_kl:np.float64(-4.779356147821545e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.40625) - self_distillation/token_reweight_w_mean:np.float64(172435.96042609704) - self_distillation/token_reweight_w_std:np.float64(2496012.8363360455) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0023111787158996) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14190766037791036) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.691222682595253) - perf/mfu/actor:np.float64(0.03488282316470846) - perf/max_memory_allocated_gb:np.float64(118.78376626968384) - perf/max_memory_reserved_gb:np.float64(124.83203125) - perf/cpu_memory_used_gb:np.float64(141.33230590820312) - actor/lr:np.float64(6e-07) - training/global_step:7 - training/epoch:0 - critic/score/mean:0.40625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0029152068309485912 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0029152068309485912 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response
(TaskRunner pid=2026458) Training Progress: 7%|▋ | 7/100 [04:22<51:17, 33.10s/it]
(TaskRunner pid=2026458) step:8 - global_seqlen/min:18363 - global_seqlen/max:28639 - global_seqlen/minmax_diff:10276 - global_seqlen/balanced_min:20588 - global_seqlen/balanced_max:27784 - global_seqlen/mean:22387.5 - actor/entropy:0.1880139708518982 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4120119512081146 - training/rollout_probs_diff_mean:0.004839833360165358 - training/rollout_probs_diff_std:0.014090316370129585 - training/rollout_actor_probs_pearson_corr:0.9977223873138428 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.52734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:0.9999743103981018 - rollout_corr/rollout_is_ratio_fraction_high:4.797911969944835e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.636241545900702e-05 - rollout_corr/rollout_is_max:3.370049476623535 - rollout_corr/rollout_is_min:0.3127705454826355 - rollout_corr/rollout_is_std:0.04152974858880043 - rollout_corr/rollout_is_eff_sample_size:0.9982780119380065 - rollout_corr/rollout_is_seq_mean:0.9999589323997498 - rollout_corr/rollout_is_seq_std:0.0027529550716280937 - rollout_corr/rollout_is_seq_max:1.010353922843933 - rollout_corr/rollout_is_seq_min:0.9923502802848816 - rollout_corr/rollout_is_seq_max_deviation:0.010353922843933105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2731395619921386) - rollout_corr/training_log_ppl:np.float64(0.23774879045959096) - rollout_corr/kl:np.float64(0.0014767409559155453) - rollout_corr/k3_kl:np.float64(0.0011382510237183396) - rollout_corr/rollout_ppl:np.float64(1.2712108385749161) - rollout_corr/rollout_log_ppl:np.float64(0.23627204724471085) - rollout_corr/log_ppl_diff:np.float64(0.0014767432148801163) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023151475324993953) - rollout_corr/log_ppl_diff_max:np.float64(0.010397493839263916) - rollout_corr/log_ppl_diff_min:np.float64(-0.009037822484970093) - rollout_corr/ppl_ratio:np.float64(1.0014815824106336) - rollout_corr/chi2_token:np.float64(0.0017209488432854414) - rollout_corr/chi2_seq:np.float64(0.5925183375366032) - actor/pg_loss:np.float64(0.00838217546697706) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010998969960382965) - actor/ppo_kl:np.float64(0.0003410603646214483) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.52734375) - self_distillation/token_reweight_w_mean:np.float64(156066.41963776737) - self_distillation/token_reweight_w_std:np.float64(2406522.8341758223) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0018263959791511) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12189624947495759) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.6973418667912483) - perf/mfu/actor:np.float64(0.03905740907932045) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(141.18081665039062) - actor/lr:np.float64(7e-07) - training/global_step:8 - training/epoch:0 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.04101495072245598 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.04101495072245598 - critic/returns/max:0.875 - critic/returns/min:-0.875 - r
(TaskRunner pid=2026458) Training Progress: 8%|▊ | 8/100 [05:40<1:12:15, 47.12s/it]
(TaskRunner pid=2026458) step:9 - global_seqlen/min:16365 - global_seqlen/max:21320 - global_seqlen/minmax_diff:4955 - global_seqlen/balanced_min:19315 - global_seqlen/balanced_max:19346 - global_seqlen/mean:19319.75 - actor/entropy:0.26203233003616333 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9982497692108154 - training/rollout_probs_diff_mean:0.0062668537721037865 - training/rollout_probs_diff_std:0.015974536538124084 - training/rollout_actor_probs_pearson_corr:0.9976486563682556 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6328125 - self_distillation/correct_sample_fraction:0.32421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6328125 - rollout_corr/rollout_is_mean:0.9996777772903442 - rollout_corr/rollout_is_ratio_fraction_high:6.202396616572514e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011164313764311373 - rollout_corr/rollout_is_max:3.62428617477417 - rollout_corr/rollout_is_min:3.9347321489913156e-07 - rollout_corr/rollout_is_std:0.047304097563028336 - rollout_corr/rollout_is_eff_sample_size:0.9977658943184634 - rollout_corr/rollout_is_seq_mean:0.9996783137321472 - rollout_corr/rollout_is_seq_std:0.0031972972210496664 - rollout_corr/rollout_is_seq_max:1.0095512866973877 - rollout_corr/rollout_is_seq_min:0.988174557685852 - rollout_corr/rollout_is_seq_max_deviation:0.01182544231414795 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3248378653079271) - rollout_corr/training_log_ppl:np.float64(0.2779389526695013) - rollout_corr/kl:np.float64(0.0020241939025922306) - rollout_corr/k3_kl:np.float64(0.0015429796273451757) - rollout_corr/rollout_ppl:np.float64(1.3221956714987755) - rollout_corr/rollout_log_ppl:np.float64(0.2759147551842034) - rollout_corr/log_ppl_diff:np.float64(0.0020241974852979183) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029990713810548186) - rollout_corr/log_ppl_diff_max:np.float64(0.053256407380104065) - rollout_corr/log_ppl_diff_min:np.float64(-0.006744384765625) - rollout_corr/ppl_ratio:np.float64(1.0020368602126837) - rollout_corr/chi2_token:np.float64(0.0018867282196879387) - rollout_corr/chi2_seq:np.float64(0.3200019830837846) - actor/pg_loss:np.float64(0.007030418026261032) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012842502446801518) - actor/ppo_kl:np.float64(0.0002998838296477402) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.32421875) - self_distillation/token_reweight_w_mean:np.float64(251775.1740568783) - self_distillation/token_reweight_w_std:np.float64(3725947.0044942726) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0048442378174514) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11786546447547153) - self_distillation/empty_target_batch:np.float64(0.3671875) - actor/grad_norm:np.float64(0.692237138748169) - perf/mfu/actor:np.float64(0.035503440773592805) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(141.21575927734375) - actor/lr:np.float64(8e-07) - training/global_step:9 - training/epoch:0 - critic/score/mean:0.32421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.32421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012389287585392594 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012389287585392594 - critic/returns/max:0.875 - critic/returns/min:-0.875 - re
(TaskRunner pid=2026458) Training Progress: 9%|▉ | 9/100 [06:11<1:04:14, 42.35s/it]
(TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 10}
(TaskRunner pid=2026458) validation generation end
(TaskRunner pid=2026458) [prompt] system
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2026458) <reasoning>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) user
(TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2026458) Please reason step by step.
(TaskRunner pid=2026458) assistant
(TaskRunner pid=2026458) <think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) </think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) [response] <reasoning>
(TaskRunner pid=2026458) The question asks which molecule is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a complex structure with a benzene ring substituted with a chlorine atom and a trifluoromethyl group (C(F)(F)F). The key part of the product is the "Cc2ccc(Cl)cc2C(F)(F)F" fragment, which suggests that the reactant must contain a similar structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Looking at the options:
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) - Option A contains a ketone group (O=C) and a nitrogen-containing ring, but it does not match the specific substituents in the product.
(TaskRunner pid=2026458) - Option B contains a chlorine atom and a nitrogen, but it does not include the trifluoromethyl group (C(F)(F)F).
(TaskRunner pid=2026458) - Option C contains a trifluoromethyl group (C(F)(F)F) and a chlorine atom, and it includes a structure that matches the "Cc2ccc(Cl)cc2" part of the product.
(TaskRunner pid=2026458) - Option D contains a trifluoromethyl group but lacks the chlorine atom in the correct position.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option C is the only one that includes both the chlorine and the trifluoromethyl group in the correct positions, matching the structure of the product. Therefore, it is the correct reactant.
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) C
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458) [ground_truth] A
(TaskRunner pid=2026458) [score] 0.0
(TaskRunner pid=2026458) [acc] 0.0
(TaskRunner pid=2026458) [pred] C
(TaskRunner pid=2026458) [incorrect_format] 1
(TaskRunner pid=2026458) [feedback]
(TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10
(TaskRunner pid=2026458) step:10 - global_seqlen/min:17490 - global_seqlen/max:20901 - global_seqlen/minmax_diff:3411 - global_seqlen/balanced_min:19357 - global_seqlen/balanced_max:19367 - global_seqlen/mean:19364.75 - actor/entropy:0.24537719786167145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744035601615906 - training/rollout_probs_diff_mean:0.005785205401480198 - training/rollout_probs_diff_std:0.014700246043503284 - training/rollout_actor_probs_pearson_corr:0.9979208707809448 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9998391270637512 - rollout_corr/rollout_is_ratio_fraction_high:1.1633588655968197e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.980152829783037e-05 - rollout_corr/rollout_is_max:2.010491132736206 - rollout_corr/rollout_is_min:0.23784410953521729 - rollout_corr/rollout_is_std:0.04395073652267456 - rollout_corr/rollout_is_eff_sample_size:0.9980714630669532 - rollout_corr/rollout_is_seq_mean:0.9998910427093506 - rollout_corr/rollout_is_seq_std:0.0027966259513050318 - rollout_corr/rollout_is_seq_max:1.0104008913040161 - rollout_corr/rollout_is_seq_min:0.9919784069061279 - rollout_corr/rollout_is_seq_max_deviation:0.010400891304016113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3026330736465752) - rollout_corr/training_log_ppl:np.float64(0.26078601760673337) - rollout_corr/kl:np.float64(0.0013049291076470126) - rollout_corr/k3_kl:np.float64(0.0011964667813799679) - rollout_corr/rollout_ppl:np.float64(1.3009595507755876) - rollout_corr/rollout_log_ppl:np.float64(0.25948108817101456) - rollout_corr/log_ppl_diff:np.float64(0.0013049294357188046) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024894755915738642) - rollout_corr/log_ppl_diff_max:np.float64(0.017253637313842773) - rollout_corr/log_ppl_diff_min:np.float64(-0.008967161178588867) - rollout_corr/ppl_ratio:np.float64(1.0013103527016938) - rollout_corr/chi2_token:np.float64(0.002327824244275689) - rollout_corr/chi2_seq:np.float64(0.852935308823362) - actor/pg_loss:np.float64(0.00642693554982543) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010866172510759498) - actor/ppo_kl:np.float64(0.00015771930741692586) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(177288.61759452336) - self_distillation/token_reweight_w_std:np.float64(2583470.6754472964) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025011864490807) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1041747338604182) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.5560944229364395) - perf/mfu/actor:np.float64(0.03574857500362935) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(141.2105484008789) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/reward/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.2097
(TaskRunner pid=2026458) Training Progress: 10%|█ | 10/100 [08:48<1:56:19, 77.55s/it]
(TaskRunner pid=2026458) step:11 - global_seqlen/min:17717 - global_seqlen/max:22105 - global_seqlen/minmax_diff:4388 - global_seqlen/balanced_min:20139 - global_seqlen/balanced_max:20427 - global_seqlen/mean:20176.875 - actor/entropy:0.28003615140914917 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2986578345298767 - training/rollout_probs_diff_mean:0.006229304242879152 - training/rollout_probs_diff_std:0.014922214671969414 - training/rollout_actor_probs_pearson_corr:0.9980378746986389 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9996895790100098 - rollout_corr/rollout_is_ratio_fraction_high:2.3571841666125692e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.071552681736648e-05 - rollout_corr/rollout_is_max:2.091331720352173 - rollout_corr/rollout_is_min:0.056948471814394 - rollout_corr/rollout_is_std:0.04615355655550957 - rollout_corr/rollout_is_eff_sample_size:0.9978729526678736 - rollout_corr/rollout_is_seq_mean:0.9996609091758728 - rollout_corr/rollout_is_seq_std:0.002769676735624671 - rollout_corr/rollout_is_seq_max:1.0097713470458984 - rollout_corr/rollout_is_seq_min:0.9914740920066833 - rollout_corr/rollout_is_seq_max_deviation:0.009771347045898438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3445568550378084) - rollout_corr/training_log_ppl:np.float64(0.2928216397704091) - rollout_corr/kl:np.float64(0.0013540790551616055) - rollout_corr/k3_kl:np.float64(0.001210083299042708) - rollout_corr/rollout_ppl:np.float64(1.3426842936314642) - rollout_corr/rollout_log_ppl:np.float64(0.2914675589709077) - rollout_corr/log_ppl_diff:np.float64(0.0013540807995013893) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025300010456703603) - rollout_corr/log_ppl_diff_max:np.float64(0.011778771877288818) - rollout_corr/log_ppl_diff_min:np.float64(-0.006693422794342041) - rollout_corr/ppl_ratio:np.float64(1.001359449699521) - rollout_corr/chi2_token:np.float64(0.0021713071037083864) - rollout_corr/chi2_seq:np.float64(0.5860715808812529) - actor/pg_loss:np.float64(0.009074105182662606) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010309692956980143) - actor/ppo_kl:np.float64(-0.00014117327255291912) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(239610.312335568) - self_distillation/token_reweight_w_std:np.float64(3316290.997419097) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019973907619715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14387543653720059) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.6877006888389587) - perf/mfu/actor:np.float64(0.037142691887104826) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9859390258789) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006922754924744368 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006922754924744368 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_le
(TaskRunner pid=2026458) Training Progress: 11%|█ | 11/100 [09:20<1:34:40, 63.83s/it]
(TaskRunner pid=2026458) step:12 - global_seqlen/min:17937 - global_seqlen/max:22358 - global_seqlen/minmax_diff:4421 - global_seqlen/balanced_min:19749 - global_seqlen/balanced_max:19763 - global_seqlen/mean:19757.25 - actor/entropy:0.2599850594997406 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29290565848350525 - training/rollout_probs_diff_mean:0.00601433590054512 - training/rollout_probs_diff_std:0.014930167235434055 - training/rollout_actor_probs_pearson_corr:0.9979494214057922 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.4609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.999984085559845 - rollout_corr/rollout_is_ratio_fraction_high:2.223754199803807e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.33563111780677e-05 - rollout_corr/rollout_is_max:2.084258556365967 - rollout_corr/rollout_is_min:0.48210135102272034 - rollout_corr/rollout_is_std:0.04539177566766739 - rollout_corr/rollout_is_eff_sample_size:0.9979434670549656 - rollout_corr/rollout_is_seq_mean:0.9999362230300903 - rollout_corr/rollout_is_seq_std:0.002637566300109029 - rollout_corr/rollout_is_seq_max:1.0081006288528442 - rollout_corr/rollout_is_seq_min:0.9917522072792053 - rollout_corr/rollout_is_seq_max_deviation:0.008247792720794678 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3254084084182978) - rollout_corr/training_log_ppl:np.float64(0.2779389991774224) - rollout_corr/kl:np.float64(0.0013704395118168122) - rollout_corr/k3_kl:np.float64(0.0011261843454803966) - rollout_corr/rollout_ppl:np.float64(1.3235342632979155) - rollout_corr/rollout_log_ppl:np.float64(0.27656855969689786) - rollout_corr/log_ppl_diff:np.float64(0.0013704394805245101) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024474224192090333) - rollout_corr/log_ppl_diff_max:np.float64(0.011240929365158081) - rollout_corr/log_ppl_diff_min:np.float64(-0.010027527809143066) - rollout_corr/ppl_ratio:np.float64(1.001375619089231) - rollout_corr/chi2_token:np.float64(0.001807245658710599) - rollout_corr/chi2_seq:np.float64(1.2864351340103894) - actor/pg_loss:np.float64(0.0068493077997118235) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010652919690983254) - actor/ppo_kl:np.float64(0.00020145679512317827) - actor/pg_clipfrac_lower:np.float64(1.669337689236272e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4609375) - self_distillation/token_reweight_w_mean:np.float64(145999.17218339257) - self_distillation/token_reweight_w_std:np.float64(2374372.460695313) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004515508189797) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1246801737579517) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.5914773046970367) - perf/mfu/actor:np.float64(0.03625096042724267) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.91592407226562) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004239031113684177 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004239031113684177 - critic/returns/max:0.875 - critic/returns/
(TaskRunner pid=2026458) Training Progress: 12%|█▏ | 12/100 [09:53<1:19:39, 54.31s/it]
(TaskRunner pid=2026458) step:13 - global_seqlen/min:17460 - global_seqlen/max:22387 - global_seqlen/minmax_diff:4927 - global_seqlen/balanced_min:19538 - global_seqlen/balanced_max:19656 - global_seqlen/mean:19562.0 - actor/entropy:0.28330478072166443 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932348847389221 - training/rollout_probs_diff_mean:0.006090556271374226 - training/rollout_probs_diff_std:0.014429626986384392 - training/rollout_actor_probs_pearson_corr:0.998181939125061 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:0.9998684525489807 - rollout_corr/rollout_is_ratio_fraction_high:2.3373224394163117e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.3373224394163117e-05 - rollout_corr/rollout_is_max:2.1884407997131348 - rollout_corr/rollout_is_min:0.4281519055366516 - rollout_corr/rollout_is_std:0.04431807994842529 - rollout_corr/rollout_is_eff_sample_size:0.9980391641302073 - rollout_corr/rollout_is_seq_mean:0.9998126029968262 - rollout_corr/rollout_is_seq_std:0.0026776245795190334 - rollout_corr/rollout_is_seq_max:1.009365439414978 - rollout_corr/rollout_is_seq_min:0.9913782477378845 - rollout_corr/rollout_is_seq_max_deviation:0.009365439414978027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3591533433645964) - rollout_corr/training_log_ppl:np.float64(0.30308022926328704) - rollout_corr/kl:np.float64(0.0012214116412607723) - rollout_corr/k3_kl:np.float64(0.0011643693818541578) - rollout_corr/rollout_ppl:np.float64(1.3574783634394407) - rollout_corr/rollout_log_ppl:np.float64(0.3018588193517644) - rollout_corr/log_ppl_diff:np.float64(0.0012214099115226418) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024031201901379973) - rollout_corr/log_ppl_diff_max:np.float64(0.011142253875732422) - rollout_corr/log_ppl_diff_min:np.float64(-0.00809323787689209) - rollout_corr/ppl_ratio:np.float64(1.0012265297118574) - rollout_corr/chi2_token:np.float64(0.0022032533306628466) - rollout_corr/chi2_seq:np.float64(1.1880777331534773) - actor/pg_loss:np.float64(0.008368437876924872) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011567205356186605) - actor/ppo_kl:np.float64(-1.6140308314760432e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(93498.56042674859) - self_distillation/token_reweight_w_std:np.float64(1510537.690864286) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999996354104951) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12338341507711448) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.620448037981987) - perf/mfu/actor:np.float64(0.036300476348807696) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.69003295898438) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003617006354033947 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003617006354033947 - critic/returns/max:0.875 - critic/returns/min:-0.875 -
(TaskRunner pid=2026458) Training Progress: 13%|█▎ | 13/100 [10:25<1:08:56, 47.54s/it]
(TaskRunner pid=2026458) step:14 - global_seqlen/min:16941 - global_seqlen/max:22560 - global_seqlen/minmax_diff:5619 - global_seqlen/balanced_min:18841 - global_seqlen/balanced_max:18844 - global_seqlen/mean:18842.75 - actor/entropy:0.2922348976135254 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5466010570526123 - training/rollout_probs_diff_mean:0.006264000199735165 - training/rollout_probs_diff_std:0.014866359531879425 - training/rollout_actor_probs_pearson_corr:0.9981194138526917 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0002233982086182 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.292350346688181e-05 - rollout_corr/rollout_is_max:1.9629987478256226 - rollout_corr/rollout_is_min:0.12186871469020844 - rollout_corr/rollout_is_std:0.04516667500138283 - rollout_corr/rollout_is_eff_sample_size:0.9979653119171001 - rollout_corr/rollout_is_seq_mean:1.0002156496047974 - rollout_corr/rollout_is_seq_std:0.00266577722504735 - rollout_corr/rollout_is_seq_max:1.0086764097213745 - rollout_corr/rollout_is_seq_min:0.9924325942993164 - rollout_corr/rollout_is_seq_max_deviation:0.008676409721374512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.356894207186997) - rollout_corr/training_log_ppl:np.float64(0.30215946782846004) - rollout_corr/kl:np.float64(0.0010239662146345752) - rollout_corr/k3_kl:np.float64(0.0012049657070747344) - rollout_corr/rollout_ppl:np.float64(1.3555219103582203) - rollout_corr/rollout_log_ppl:np.float64(0.3011355001362972) - rollout_corr/log_ppl_diff:np.float64(0.0010239676921628416) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023661727900616825) - rollout_corr/log_ppl_diff_max:np.float64(0.017396926879882812) - rollout_corr/log_ppl_diff_min:np.float64(-0.007369279861450195) - rollout_corr/ppl_ratio:np.float64(1.0010292823426425) - rollout_corr/chi2_token:np.float64(0.0027081489097326994) - rollout_corr/chi2_seq:np.float64(1.2070772459264845) - actor/pg_loss:np.float64(0.00797818333376199) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010150298585358541) - actor/ppo_kl:np.float64(0.00015036190526984683) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.453125) - self_distillation/token_reweight_w_mean:np.float64(119982.02733938815) - self_distillation/token_reweight_w_std:np.float64(1933813.0637409594) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014771707355976) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1388165877724532) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.6455883979797363) - perf/mfu/actor:np.float64(0.034641753809947065) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.4944953918457) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026480345986783504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026480345986783504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:321.3984
(TaskRunner pid=2026458) Training Progress: 14%|█▍ | 14/100 [10:55<1:00:33, 42.25s/it]
(TaskRunner pid=2026458) step:15 - global_seqlen/min:17798 - global_seqlen/max:24465 - global_seqlen/minmax_diff:6667 - global_seqlen/balanced_min:20570 - global_seqlen/balanced_max:20574 - global_seqlen/mean:20572.75 - actor/entropy:0.28748324513435364 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564783573150635 - training/rollout_probs_diff_mean:0.005869686137884855 - training/rollout_probs_diff_std:0.014366100542247295 - training/rollout_actor_probs_pearson_corr:0.9982949495315552 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0001139640808105 - rollout_corr/rollout_is_ratio_fraction_high:5.4864267440279946e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3891413952223957e-05 - rollout_corr/rollout_is_max:3.0497446060180664 - rollout_corr/rollout_is_min:0.38640856742858887 - rollout_corr/rollout_is_std:0.045057009905576706 - rollout_corr/rollout_is_eff_sample_size:0.9979744538008674 - rollout_corr/rollout_is_seq_mean:1.0000189542770386 - rollout_corr/rollout_is_seq_std:0.002783029805868864 - rollout_corr/rollout_is_seq_max:1.0093011856079102 - rollout_corr/rollout_is_seq_min:0.9919822812080383 - rollout_corr/rollout_is_seq_max_deviation:0.009301185607910156 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3641661452129483) - rollout_corr/training_log_ppl:np.float64(0.3061258476227522) - rollout_corr/kl:np.float64(0.0008984394618103408) - rollout_corr/k3_kl:np.float64(0.0010934443635051139) - rollout_corr/rollout_ppl:np.float64(1.3629517373628914) - rollout_corr/rollout_log_ppl:np.float64(0.3052274066139944) - rollout_corr/log_ppl_diff:np.float64(0.0008984410087577999) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023149027838371694) - rollout_corr/log_ppl_diff_max:np.float64(0.015007197856903076) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058860182762146) - rollout_corr/ppl_ratio:np.float64(1.00090327905491) - rollout_corr/chi2_token:np.float64(0.0026275489944964647) - rollout_corr/chi2_seq:np.float64(1.554575996240601) - actor/pg_loss:np.float64(0.006172901834361255) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010921142766164849) - actor/ppo_kl:np.float64(-0.00014767027690254508) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(168825.68510642461) - self_distillation/token_reweight_w_std:np.float64(2411026.6848952193) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999142882879823) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13169058857602067) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.5341654643416405) - perf/mfu/actor:np.float64(0.03786336781112096) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.43387985229492) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036114403046667576 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036114403046667576 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_le
(TaskRunner pid=2026458) Training Progress: 15%|█▌ | 15/100 [11:26<55:04, 38.87s/it]
(TaskRunner pid=2026458) step:16 - global_seqlen/min:17436 - global_seqlen/max:21462 - global_seqlen/minmax_diff:4026 - global_seqlen/balanced_min:19472 - global_seqlen/balanced_max:19474 - global_seqlen/mean:19473.375 - actor/entropy:0.29451751708984375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34674251079559326 - training/rollout_probs_diff_mean:0.006398364901542664 - training/rollout_probs_diff_std:0.014944251626729965 - training/rollout_actor_probs_pearson_corr:0.9980317950248718 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.64453125 - self_distillation/correct_sample_fraction:0.42578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.64453125 - rollout_corr/rollout_is_mean:1.000139832496643 - rollout_corr/rollout_is_ratio_fraction_high:3.57470526068937e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001429882104275748 - rollout_corr/rollout_is_max:2.075308322906494 - rollout_corr/rollout_is_min:0.33335795998573303 - rollout_corr/rollout_is_std:0.04721960052847862 - rollout_corr/rollout_is_eff_sample_size:0.9977759819851924 - rollout_corr/rollout_is_seq_mean:1.0001648664474487 - rollout_corr/rollout_is_seq_std:0.002871463308110833 - rollout_corr/rollout_is_seq_max:1.0102380514144897 - rollout_corr/rollout_is_seq_min:0.9917946457862854 - rollout_corr/rollout_is_seq_max_deviation:0.010238051414489746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36143590323627) - rollout_corr/training_log_ppl:np.float64(0.3041792264266405) - rollout_corr/kl:np.float64(0.0010337699030245062) - rollout_corr/k3_kl:np.float64(0.0012385786424147227) - rollout_corr/rollout_ppl:np.float64(1.3600022280588746) - rollout_corr/rollout_log_ppl:np.float64(0.30314545703004114) - rollout_corr/log_ppl_diff:np.float64(0.0010337693965993822) - rollout_corr/log_ppl_abs_diff:np.float64(0.002399776072707027) - rollout_corr/log_ppl_diff_max:np.float64(0.011399328708648682) - rollout_corr/log_ppl_diff_min:np.float64(-0.006810277700424194) - rollout_corr/ppl_ratio:np.float64(1.0010383201297373) - rollout_corr/chi2_token:np.float64(0.0029240259900689125) - rollout_corr/chi2_seq:np.float64(1.0353804978076369) - actor/pg_loss:np.float64(0.007192879682406783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010274524142914743) - actor/ppo_kl:np.float64(4.0873823671461196e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.64453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.42578125) - self_distillation/token_reweight_w_mean:np.float64(90371.76209130394) - self_distillation/token_reweight_w_std:np.float64(1389709.4737052019) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000855000689626) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11968303663888946) - self_distillation/empty_target_batch:np.float64(0.35546875) - actor/grad_norm:np.float64(0.6043514311313629) - perf/mfu/actor:np.float64(0.03566211221390109) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.67253494262695) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006435959134250879 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006435959134250879 - critic/returns/max:0.875 - critic/returns/min:-0.875 -
(TaskRunner pid=2026458) Training Progress: 16%|█▌ | 16/100 [11:56<50:48, 36.29s/it]
(TaskRunner pid=2026458) step:17 - global_seqlen/min:19903 - global_seqlen/max:25273 - global_seqlen/minmax_diff:5370 - global_seqlen/balanced_min:22184 - global_seqlen/balanced_max:22190 - global_seqlen/mean:22187.375 - actor/entropy:0.28030383586883545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4096241295337677 - training/rollout_probs_diff_mean:0.00557515537366271 - training/rollout_probs_diff_std:0.014061576686799526 - training/rollout_actor_probs_pearson_corr:0.9982938766479492 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.640625 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.640625 - rollout_corr/rollout_is_mean:1.000001311302185 - rollout_corr/rollout_is_ratio_fraction_high:2.1693151211366057e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.4232874390436336e-05 - rollout_corr/rollout_is_max:3.007925271987915 - rollout_corr/rollout_is_min:0.3316154181957245 - rollout_corr/rollout_is_std:0.0431625172495842 - rollout_corr/rollout_is_eff_sample_size:0.9981403427818611 - rollout_corr/rollout_is_seq_mean:1.0001276731491089 - rollout_corr/rollout_is_seq_std:0.002614374505355954 - rollout_corr/rollout_is_seq_max:1.0077310800552368 - rollout_corr/rollout_is_seq_min:0.9923279881477356 - rollout_corr/rollout_is_seq_max_deviation:0.007731080055236816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.381326341535896) - rollout_corr/training_log_ppl:np.float64(0.3175459819030948) - rollout_corr/kl:np.float64(0.0007249800520758853) - rollout_corr/k3_kl:np.float64(0.0011385135364321286) - rollout_corr/rollout_ppl:np.float64(1.3803315898403525) - rollout_corr/rollout_log_ppl:np.float64(0.316821001295466) - rollout_corr/log_ppl_diff:np.float64(0.0007249806076288223) - rollout_corr/log_ppl_abs_diff:np.float64(0.00215923844370991) - rollout_corr/log_ppl_diff_max:np.float64(0.00994020700454712) - rollout_corr/log_ppl_diff_min:np.float64(-0.007171809673309326) - rollout_corr/ppl_ratio:np.float64(1.0007290567737073) - rollout_corr/chi2_token:np.float64(0.003144999034702778) - rollout_corr/chi2_seq:np.float64(0.9032414378598332) - actor/pg_loss:np.float64(0.0055801779963076115) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008305092796945246) - actor/ppo_kl:np.float64(-0.00022399483597723702) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.640625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(213885.15984977316) - self_distillation/token_reweight_w_std:np.float64(3187382.02780142) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009852752555162) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11256717666401528) - self_distillation/empty_target_batch:np.float64(0.359375) - actor/grad_norm:np.float64(0.5271423608064651) - perf/mfu/actor:np.float64(0.04032452755142399) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.51458740234375) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0036620749160647392 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0036620749160647392 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_
(TaskRunner pid=2026458) Training Progress: 17%|█▋ | 17/100 [12:29<48:40, 35.18s/it]
(TaskRunner pid=2026458) step:18 - global_seqlen/min:17147 - global_seqlen/max:20375 - global_seqlen/minmax_diff:3228 - global_seqlen/balanced_min:18455 - global_seqlen/balanced_max:18458 - global_seqlen/mean:18457.0 - actor/entropy:0.3130648136138916 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772509455680847 - training/rollout_probs_diff_mean:0.0063330200500786304 - training/rollout_probs_diff_std:0.014608096331357956 - training/rollout_actor_probs_pearson_corr:0.9982773065567017 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:0.9997703433036804 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.43533785478212e-05 - rollout_corr/rollout_is_max:1.9900730848312378 - rollout_corr/rollout_is_min:0.4134674072265625 - rollout_corr/rollout_is_std:0.04487474635243416 - rollout_corr/rollout_is_eff_sample_size:0.9979895323387393 - rollout_corr/rollout_is_seq_mean:0.9998025298118591 - rollout_corr/rollout_is_seq_std:0.002652442781254649 - rollout_corr/rollout_is_seq_max:1.0070677995681763 - rollout_corr/rollout_is_seq_min:0.9913365244865417 - rollout_corr/rollout_is_seq_max_deviation:0.008663475513458252 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3969450276345015) - rollout_corr/training_log_ppl:np.float64(0.3299181046313606) - rollout_corr/kl:np.float64(0.001080212333377517) - rollout_corr/k3_kl:np.float64(0.0011240256234259505) - rollout_corr/rollout_ppl:np.float64(1.3953958731144667) - rollout_corr/rollout_log_ppl:np.float64(0.32883789180777967) - rollout_corr/log_ppl_diff:np.float64(0.0010802128235809505) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023538178065791726) - rollout_corr/log_ppl_diff_max:np.float64(0.011510014533996582) - rollout_corr/log_ppl_diff_min:np.float64(-0.009586125612258911) - rollout_corr/ppl_ratio:np.float64(1.0010849123355001) - rollout_corr/chi2_token:np.float64(0.0023459065705537796) - rollout_corr/chi2_seq:np.float64(0.5731152268126607) - actor/pg_loss:np.float64(0.007650406565517187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011541137814674585) - actor/ppo_kl:np.float64(-0.00018551408525979696) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(258180.61296664365) - self_distillation/token_reweight_w_std:np.float64(3445719.439609512) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022555205505341) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15918678505113348) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.6539853662252426) - perf/mfu/actor:np.float64(0.033737344150768206) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.57009506225586) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005579437594860792 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005579437594860792 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:303.5
(TaskRunner pid=2026458) Training Progress: 18%|█▊ | 18/100 [12:59<46:05, 33.73s/it]
(TaskRunner pid=2026458) step:19 - global_seqlen/min:16712 - global_seqlen/max:22903 - global_seqlen/minmax_diff:6191 - global_seqlen/balanced_min:19739 - global_seqlen/balanced_max:19742 - global_seqlen/mean:19740.875 - actor/entropy:0.30067509412765503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2716989815235138 - training/rollout_probs_diff_mean:0.006011276505887508 - training/rollout_probs_diff_std:0.014068786054849625 - training/rollout_actor_probs_pearson_corr:0.9983283281326294 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0000412464141846 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.48191299760947e-05 - rollout_corr/rollout_is_max:1.967125415802002 - rollout_corr/rollout_is_min:0.3074234426021576 - rollout_corr/rollout_is_std:0.04442286118865013 - rollout_corr/rollout_is_eff_sample_size:0.9980306147615733 - rollout_corr/rollout_is_seq_mean:1.0000803470611572 - rollout_corr/rollout_is_seq_std:0.0029041871894150972 - rollout_corr/rollout_is_seq_max:1.0110079050064087 - rollout_corr/rollout_is_seq_min:0.9903981685638428 - rollout_corr/rollout_is_seq_max_deviation:0.011007905006408691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.375831812620163) - rollout_corr/training_log_ppl:np.float64(0.31493956671329215) - rollout_corr/kl:np.float64(0.0009361723424987645) - rollout_corr/k3_kl:np.float64(0.00108607468507671) - rollout_corr/rollout_ppl:np.float64(1.3745251446962357) - rollout_corr/rollout_log_ppl:np.float64(0.31400339302490465) - rollout_corr/log_ppl_diff:np.float64(0.0009361736883874983) - rollout_corr/log_ppl_abs_diff:np.float64(0.002310327283339575) - rollout_corr/log_ppl_diff_max:np.float64(0.008226662874221802) - rollout_corr/log_ppl_diff_min:np.float64(-0.010857582092285156) - rollout_corr/ppl_ratio:np.float64(1.000940806698054) - rollout_corr/chi2_token:np.float64(0.002460781019181013) - rollout_corr/chi2_seq:np.float64(1.263379932148382) - actor/pg_loss:np.float64(0.005378377623856068) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007215913469735824) - actor/ppo_kl:np.float64(-2.377988029422795e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(92620.61167126987) - self_distillation/token_reweight_w_std:np.float64(1450653.645121337) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001238340511918) - self_distillation/token_reweight_w_clip_frac:np.float64(0.096618483774364) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.5270146429538727) - perf/mfu/actor:np.float64(0.03604524222111994) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.61618423461914) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024989761877804995 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024989761877804995 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:314.77734375
(TaskRunner pid=2026458) Training Progress: 19%|█▉ | 19/100 [13:30<44:19, 32.83s/it]
(TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20}
(TaskRunner pid=2026458) validation generation end
(TaskRunner pid=2026458) [prompt] system
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2026458) <reasoning>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) user
(TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2026458) Please reason step by step.
(TaskRunner pid=2026458) assistant
(TaskRunner pid=2026458) <think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) </think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) [response] <reasoning>
(TaskRunner pid=2026458) The question asks which molecule is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a structure with a substituted benzene ring (with a Cl group), a trifluoromethyl group (C(F)(F)F), and a nitrogen-containing amine group.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Looking at the options:
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) - Option A contains a structure with a substituted benzene ring, a nitrogen atom, and a carbonyl group. It also includes the Cl and trifluoromethyl groups, which match the product's structure. However, it has an extra nitrogen and a different arrangement that may not be the correct reactant.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) - Option B has a similar structure but with a different arrangement of substituents and a different nitrogen configuration, which does not match the product's structure exactly.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) - Option C includes a substituted benzene ring, a carbonyl group, and a different arrangement of substituents. It includes a hydroxyl group and a different type of substitution, which does not match the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) - Option D has a substituted benzene ring, a carbonyl group, and a different arrangement of substituents. It includes a different type of substitution and does not match the product's structure exactly.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option A is the only one that matches the structure of the product, including the correct substituents and arrangement of atoms. Therefore, it is the correct reactant.
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458) [ground_truth] A
(TaskRunner pid=2026458) [score] 1.0
(TaskRunner pid=2026458) [acc] 1.0
(TaskRunner pid=2026458) [pred] A
(TaskRunner pid=2026458) [incorrect_format] 1
(TaskRunner pid=2026458) [feedback]
(TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20
(WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor
(TaskRunner pid=2026458) step:20 - global_seqlen/min:17486 - global_seqlen/max:22655 - global_seqlen/minmax_diff:5169 - global_seqlen/balanced_min:20031 - global_seqlen/balanced_max:20228 - global_seqlen/mean:20058.625 - actor/entropy:0.297139972448349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3528301417827606 - training/rollout_probs_diff_mean:0.006166450679302216 - training/rollout_probs_diff_std:0.014436931349337101 - training/rollout_actor_probs_pearson_corr:0.9982419013977051 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.000038743019104 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.690288890036754e-05 - rollout_corr/rollout_is_max:1.9574382305145264 - rollout_corr/rollout_is_min:0.3535598814487457 - rollout_corr/rollout_is_std:0.04514159634709358 - rollout_corr/rollout_is_eff_sample_size:0.9979664991652689 - rollout_corr/rollout_is_seq_mean:1.0000238418579102 - rollout_corr/rollout_is_seq_std:0.0029090906027704477 - rollout_corr/rollout_is_seq_max:1.0110317468643188 - rollout_corr/rollout_is_seq_min:0.9888052940368652 - rollout_corr/rollout_is_seq_max_deviation:0.011194705963134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3683372386731207) - rollout_corr/training_log_ppl:np.float64(0.31030508255935274) - rollout_corr/kl:np.float64(0.0011023225397535086) - rollout_corr/k3_kl:np.float64(0.001434143366623175) - rollout_corr/rollout_ppl:np.float64(1.3668043981306255) - rollout_corr/rollout_log_ppl:np.float64(0.30920275836251676) - rollout_corr/log_ppl_diff:np.float64(0.0011023241968359798) - rollout_corr/log_ppl_abs_diff:np.float64(0.002514752763090655) - rollout_corr/log_ppl_diff_max:np.float64(0.0131492018699646) - rollout_corr/log_ppl_diff_min:np.float64(-0.008862733840942383) - rollout_corr/ppl_ratio:np.float64(1.0011082254350185) - rollout_corr/chi2_token:np.float64(0.0035093913320451975) - rollout_corr/chi2_seq:np.float64(1.2149556565564126) - actor/pg_loss:np.float64(0.009395288652740419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015388868646368792) - actor/ppo_kl:np.float64(4.407900779046514e-05) - actor/pg_clipfrac_lower:np.float64(3.007869418070186e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(178377.74242206803) - self_distillation/token_reweight_w_std:np.float64(2601166.713610011) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996962018776685) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1573005250247661) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.68055609613657) - perf/mfu/actor:np.float64(0.03693197062950951) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.50461959838867) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21773506231934
(TaskRunner pid=2026458) Training Progress: 20%|██ | 20/100 [16:02<1:31:32, 68.66s/it]
(TaskRunner pid=2026458) step:21 - global_seqlen/min:17387 - global_seqlen/max:23009 - global_seqlen/minmax_diff:5622 - global_seqlen/balanced_min:19296 - global_seqlen/balanced_max:19303 - global_seqlen/mean:19300.625 - actor/entropy:0.2956748902797699 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42548468708992004 - training/rollout_probs_diff_mean:0.005995167884975672 - training/rollout_probs_diff_std:0.014325084164738655 - training/rollout_actor_probs_pearson_corr:0.9982481002807617 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:0.999878466129303 - rollout_corr/rollout_is_ratio_fraction_high:1.2208670341351535e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.325202022911981e-05 - rollout_corr/rollout_is_max:2.6692886352539062 - rollout_corr/rollout_is_min:0.4042225778102875 - rollout_corr/rollout_is_std:0.044611647725105286 - rollout_corr/rollout_is_eff_sample_size:0.9980132789914753 - rollout_corr/rollout_is_seq_mean:0.9997992515563965 - rollout_corr/rollout_is_seq_std:0.0029755644500255585 - rollout_corr/rollout_is_seq_max:1.0099884271621704 - rollout_corr/rollout_is_seq_min:0.9911137819290161 - rollout_corr/rollout_is_seq_max_deviation:0.00998842716217041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3673860565759242) - rollout_corr/training_log_ppl:np.float64(0.3087902419210877) - rollout_corr/kl:np.float64(0.002253606636003269) - rollout_corr/k3_kl:np.float64(0.0017760526020538236) - rollout_corr/rollout_ppl:np.float64(1.36424843640998) - rollout_corr/rollout_log_ppl:np.float64(0.30653663334669545) - rollout_corr/log_ppl_diff:np.float64(0.002253608574392274) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032565151050221175) - rollout_corr/log_ppl_diff_max:np.float64(0.021895110607147217) - rollout_corr/log_ppl_diff_min:np.float64(-0.006019324064254761) - rollout_corr/ppl_ratio:np.float64(1.002263783942908) - rollout_corr/chi2_token:np.float64(0.002287261188030243) - rollout_corr/chi2_seq:np.float64(0.5934691540896893) - actor/pg_loss:np.float64(0.010032993159256876) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001869809015715873) - actor/ppo_kl:np.float64(0.000984730247550747) - actor/pg_clipfrac_lower:np.float64(2.9235126021376345e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(353075.78875684226) - self_distillation/token_reweight_w_std:np.float64(4698832.619528098) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0033803705591708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17246504218201153) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.7826535254716873) - perf/mfu/actor:np.float64(0.03582340340916603) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.28646850585938) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004340182524174452 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004340182524174452 - critic/returns/max:0.875 - critic/returns/min:-0.875 - re
(TaskRunner pid=2026458) Training Progress: 21%|██ | 21/100 [16:33<1:15:19, 57.20s/it]
(TaskRunner pid=2026458) step:22 - global_seqlen/min:15143 - global_seqlen/max:18708 - global_seqlen/minmax_diff:3565 - global_seqlen/balanced_min:16965 - global_seqlen/balanced_max:16966 - global_seqlen/mean:16965.375 - actor/entropy:0.31038597226142883 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2530360817909241 - training/rollout_probs_diff_mean:0.0064188591204583645 - training/rollout_probs_diff_std:0.014548459090292454 - training/rollout_actor_probs_pearson_corr:0.9982183575630188 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:1.0001835823059082 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.267485928721726e-05 - rollout_corr/rollout_is_max:1.9917649030685425 - rollout_corr/rollout_is_min:0.3914262652397156 - rollout_corr/rollout_is_std:0.0448860339820385 - rollout_corr/rollout_is_eff_sample_size:0.9979898885301821 - rollout_corr/rollout_is_seq_mean:1.0002237558364868 - rollout_corr/rollout_is_seq_std:0.0030402897391468287 - rollout_corr/rollout_is_seq_max:1.0082043409347534 - rollout_corr/rollout_is_seq_min:0.9909875988960266 - rollout_corr/rollout_is_seq_max_deviation:0.009012401103973389 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3948297398164868) - rollout_corr/training_log_ppl:np.float64(0.32873041287530214) - rollout_corr/kl:np.float64(0.0012066373147963105) - rollout_corr/k3_kl:np.float64(0.0013965993947522293) - rollout_corr/rollout_ppl:np.float64(1.3931263061240315) - rollout_corr/rollout_log_ppl:np.float64(0.32752377312863246) - rollout_corr/log_ppl_diff:np.float64(0.0012066397466696799) - rollout_corr/log_ppl_abs_diff:np.float64(0.002761501877103001) - rollout_corr/log_ppl_diff_max:np.float64(0.011306077241897583) - rollout_corr/log_ppl_diff_min:np.float64(-0.009509235620498657) - rollout_corr/ppl_ratio:np.float64(1.0012132688425481) - rollout_corr/chi2_token:np.float64(0.003218145342543721) - rollout_corr/chi2_seq:np.float64(1.0662864169571549) - actor/pg_loss:np.float64(0.006237241672351956) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020408303180374787) - actor/ppo_kl:np.float64(0.0003451099556370796) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(326844.6732725387) - self_distillation/token_reweight_w_std:np.float64(4298094.462517299) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025643010158092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1402466203435324) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.6192770302295685) - perf/mfu/actor:np.float64(0.03161979996245398) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.7820701599121) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003102818038314581 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003102818038314581 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.60546875 -
(TaskRunner pid=2026458) Training Progress: 22%|██▏ | 22/100 [17:02<1:03:27, 48.81s/it]
(TaskRunner pid=2026458) step:23 - global_seqlen/min:15206 - global_seqlen/max:25332 - global_seqlen/minmax_diff:10126 - global_seqlen/balanced_min:18411 - global_seqlen/balanced_max:25520 - global_seqlen/mean:19300.375 - actor/entropy:0.264564573764801 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4770219326019287 - training/rollout_probs_diff_mean:0.0052871485240757465 - training/rollout_probs_diff_std:0.013520004227757454 - training/rollout_actor_probs_pearson_corr:0.998356282711029 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999374747276306 - rollout_corr/rollout_is_ratio_fraction_high:3.601138087105937e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.0018966905772686e-05 - rollout_corr/rollout_is_max:2.1046066284179688 - rollout_corr/rollout_is_min:0.36882883310317993 - rollout_corr/rollout_is_std:0.041905514895915985 - rollout_corr/rollout_is_eff_sample_size:0.9982467687554272 - rollout_corr/rollout_is_seq_mean:0.9998911023139954 - rollout_corr/rollout_is_seq_std:0.0030532306991517544 - rollout_corr/rollout_is_seq_max:1.0095295906066895 - rollout_corr/rollout_is_seq_min:0.9908995628356934 - rollout_corr/rollout_is_seq_max_deviation:0.009529590606689453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3668543784879148) - rollout_corr/training_log_ppl:np.float64(0.30833037884258374) - rollout_corr/kl:np.float64(0.0014712483040568713) - rollout_corr/k3_kl:np.float64(0.0014252641368557306) - rollout_corr/rollout_ppl:np.float64(1.364856572356075) - rollout_corr/rollout_log_ppl:np.float64(0.30685912866283616) - rollout_corr/log_ppl_diff:np.float64(0.0014712501797475852) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029776765513815917) - rollout_corr/log_ppl_diff_max:np.float64(0.01450282335281372) - rollout_corr/log_ppl_diff_min:np.float64(-0.008856803178787231) - rollout_corr/ppl_ratio:np.float64(1.0014787362888455) - rollout_corr/chi2_token:np.float64(0.003022435586899519) - rollout_corr/chi2_seq:np.float64(1.3533644466660917) - actor/pg_loss:np.float64(0.007422412978485227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020335467470431468) - actor/ppo_kl:np.float64(0.00035265562636155323) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(345985.51773530385) - self_distillation/token_reweight_w_std:np.float64(4731856.136310147) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988346283789724) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16046884056413546) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.7161731868982315) - perf/mfu/actor:np.float64(0.03389860046243484) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1311912536621) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008347137831151485 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008347137831151485 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_len
(TaskRunner pid=2026458) Training Progress: 23%|██▎ | 23/100 [18:17<1:12:52, 56.79s/it]
(TaskRunner pid=2026458) step:24 - global_seqlen/min:16231 - global_seqlen/max:19060 - global_seqlen/minmax_diff:2829 - global_seqlen/balanced_min:17704 - global_seqlen/balanced_max:17708 - global_seqlen/mean:17706.25 - actor/entropy:0.29770269989967346 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5271099209785461 - training/rollout_probs_diff_mean:0.0061988383531570435 - training/rollout_probs_diff_std:0.014574181288480759 - training/rollout_actor_probs_pearson_corr:0.9981999397277832 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.000082015991211 - rollout_corr/rollout_is_ratio_fraction_high:2.8622949685086496e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011449179874034598 - rollout_corr/rollout_is_max:2.555720567703247 - rollout_corr/rollout_is_min:0.15318159759044647 - rollout_corr/rollout_is_std:0.04580353945493698 - rollout_corr/rollout_is_eff_sample_size:0.997906784084707 - rollout_corr/rollout_is_seq_mean:1.000064492225647 - rollout_corr/rollout_is_seq_std:0.0029369699768722057 - rollout_corr/rollout_is_seq_max:1.0110825300216675 - rollout_corr/rollout_is_seq_min:0.9918853640556335 - rollout_corr/rollout_is_seq_max_deviation:0.01108253002166748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36986547941342) - rollout_corr/training_log_ppl:np.float64(0.31092395351151936) - rollout_corr/kl:np.float64(0.0015822704315340275) - rollout_corr/k3_kl:np.float64(0.0015203862882344765) - rollout_corr/rollout_ppl:np.float64(1.3676151768304408) - rollout_corr/rollout_log_ppl:np.float64(0.3093416829360649) - rollout_corr/log_ppl_diff:np.float64(0.0015822705754544586) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029158030229154974) - rollout_corr/log_ppl_diff_max:np.float64(0.013341128826141357) - rollout_corr/log_ppl_diff_min:np.float64(-0.009624361991882324) - rollout_corr/ppl_ratio:np.float64(1.0015899618156254) - rollout_corr/chi2_token:np.float64(0.0028670839965343475) - rollout_corr/chi2_seq:np.float64(0.6717999817337841) - actor/pg_loss:np.float64(0.0065518481424078345) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001698189102626202) - actor/ppo_kl:np.float64(0.000539918119613958) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(259653.62712082197) - self_distillation/token_reweight_w_std:np.float64(3592355.994360844) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997991549316794) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1357375134830363) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.6166152656078339) - perf/mfu/actor:np.float64(0.03283040904184733) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1153450012207) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003980379085987806 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003980379085987806 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_
(TaskRunner pid=2026458) Training Progress: 24%|██▍ | 24/100 [18:46<1:01:25, 48.49s/it]
(TaskRunner pid=2026458) step:25 - global_seqlen/min:15795 - global_seqlen/max:22433 - global_seqlen/minmax_diff:6638 - global_seqlen/balanced_min:18523 - global_seqlen/balanced_max:18527 - global_seqlen/mean:18525.875 - actor/entropy:0.2758452296257019 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25533485412597656 - training/rollout_probs_diff_mean:0.005769852548837662 - training/rollout_probs_diff_std:0.013884530402719975 - training/rollout_actor_probs_pearson_corr:0.9983004331588745 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.78515625 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78515625 - rollout_corr/rollout_is_mean:1.0001115798950195 - rollout_corr/rollout_is_ratio_fraction_high:1.414367125107674e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.828734250215348e-05 - rollout_corr/rollout_is_max:2.6026298999786377 - rollout_corr/rollout_is_min:0.3732539415359497 - rollout_corr/rollout_is_std:0.04336366429924965 - rollout_corr/rollout_is_eff_sample_size:0.9981233594863631 - rollout_corr/rollout_is_seq_mean:1.000120759010315 - rollout_corr/rollout_is_seq_std:0.003118946449831128 - rollout_corr/rollout_is_seq_max:1.0095044374465942 - rollout_corr/rollout_is_seq_min:0.990989625453949 - rollout_corr/rollout_is_seq_max_deviation:0.009504437446594238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3535170801915228) - rollout_corr/training_log_ppl:np.float64(0.2983562831650488) - rollout_corr/kl:np.float64(0.000989430469498842) - rollout_corr/k3_kl:np.float64(0.0011073322453967194) - rollout_corr/rollout_ppl:np.float64(1.3521247510798275) - rollout_corr/rollout_log_ppl:np.float64(0.29736685164971277) - rollout_corr/log_ppl_diff:np.float64(0.0009894315153360367) - rollout_corr/log_ppl_abs_diff:np.float64(0.002447564620524645) - rollout_corr/log_ppl_diff_max:np.float64(0.01045176386833191) - rollout_corr/log_ppl_diff_min:np.float64(-0.008554220199584961) - rollout_corr/ppl_ratio:np.float64(1.0009945430792868) - rollout_corr/chi2_token:np.float64(0.002472628140822053) - rollout_corr/chi2_seq:np.float64(0.6732058441266418) - actor/pg_loss:np.float64(0.005739172571338713) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010865399485737726) - actor/ppo_kl:np.float64(0.0001121951680964628) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(393425.2932063611) - self_distillation/token_reweight_w_std:np.float64(5274741.3204780845) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0018663464579731) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12906738638412207) - self_distillation/empty_target_batch:np.float64(0.21484375) - actor/grad_norm:np.float64(0.6542618125677109) - perf/mfu/actor:np.float64(0.034142301102186606) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1241111755371) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005874927621334791 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005874927621334791 - critic/returns/max:0.875 - critic/returns/min:-0.875 - res
(TaskRunner pid=2026458) Training Progress: 25%|██▌ | 25/100 [19:16<53:40, 42.94s/it]
(TaskRunner pid=2026458) step:26 - global_seqlen/min:16232 - global_seqlen/max:20091 - global_seqlen/minmax_diff:3859 - global_seqlen/balanced_min:18014 - global_seqlen/balanced_max:18018 - global_seqlen/mean:18016.5 - actor/entropy:0.28236204385757446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797455430030823 - training/rollout_probs_diff_mean:0.005817539524286985 - training/rollout_probs_diff_std:0.014095457270741463 - training/rollout_actor_probs_pearson_corr:0.9982751607894897 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000920295715332 - rollout_corr/rollout_is_ratio_fraction_high:1.4651584933744743e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.790950960246846e-05 - rollout_corr/rollout_is_max:2.0633111000061035 - rollout_corr/rollout_is_min:0.2806716561317444 - rollout_corr/rollout_is_std:0.044501952826976776 - rollout_corr/rollout_is_eff_sample_size:0.9980238466152811 - rollout_corr/rollout_is_seq_mean:1.0001037120819092 - rollout_corr/rollout_is_seq_std:0.0033811419270932674 - rollout_corr/rollout_is_seq_max:1.010282039642334 - rollout_corr/rollout_is_seq_min:0.9834394454956055 - rollout_corr/rollout_is_seq_max_deviation:0.01656055450439453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3481695111840963) - rollout_corr/training_log_ppl:np.float64(0.294715576863382) - rollout_corr/kl:np.float64(0.00122398861193318) - rollout_corr/k3_kl:np.float64(0.0011840371912512637) - rollout_corr/rollout_ppl:np.float64(1.3464941019192338) - rollout_corr/rollout_log_ppl:np.float64(0.2934915872756392) - rollout_corr/log_ppl_diff:np.float64(0.0012239895877428353) - rollout_corr/log_ppl_abs_diff:np.float64(0.002889484982006252) - rollout_corr/log_ppl_diff_max:np.float64(0.026334106922149658) - rollout_corr/log_ppl_diff_min:np.float64(-0.00859677791595459) - rollout_corr/ppl_ratio:np.float64(1.0012316501233727) - rollout_corr/chi2_token:np.float64(0.002297403523698449) - rollout_corr/chi2_seq:np.float64(1.2100839058402926) - actor/pg_loss:np.float64(0.004832616308704019) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006622324299314641) - actor/ppo_kl:np.float64(0.00027824501646955824) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(57253.49309117114) - self_distillation/token_reweight_w_std:np.float64(780711.6470303052) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9960764090064913) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10869685839861631) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.5899917781352997) - perf/mfu/actor:np.float64(0.03340075709542584) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.18029022216797) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0030969788786023855 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0030969788786023855 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/m
(TaskRunner pid=2026458) Training Progress: 26%|██▌ | 26/100 [19:46<47:51, 38.80s/it]
(TaskRunner pid=2026458) step:27 - global_seqlen/min:14932 - global_seqlen/max:19899 - global_seqlen/minmax_diff:4967 - global_seqlen/balanced_min:17014 - global_seqlen/balanced_max:17065 - global_seqlen/mean:17022.125 - actor/entropy:0.27002352476119995 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583366632461548 - training/rollout_probs_diff_mean:0.005672288592904806 - training/rollout_probs_diff_std:0.013904700987040997 - training/rollout_actor_probs_pearson_corr:0.9982785582542419 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:1.0000253915786743 - rollout_corr/rollout_is_ratio_fraction_high:5.697275264537893e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4243188161344733e-05 - rollout_corr/rollout_is_max:2.935737371444702 - rollout_corr/rollout_is_min:0.4851767420768738 - rollout_corr/rollout_is_std:0.04337328299880028 - rollout_corr/rollout_is_eff_sample_size:0.9981222906269569 - rollout_corr/rollout_is_seq_mean:0.9999536275863647 - rollout_corr/rollout_is_seq_std:0.002799020381644368 - rollout_corr/rollout_is_seq_max:1.0089412927627563 - rollout_corr/rollout_is_seq_min:0.991137683391571 - rollout_corr/rollout_is_seq_max_deviation:0.008941292762756348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3306819731369615) - rollout_corr/training_log_ppl:np.float64(0.28223902196623385) - rollout_corr/kl:np.float64(0.0010320211754617503) - rollout_corr/k3_kl:np.float64(0.0012100190330102123) - rollout_corr/rollout_ppl:np.float64(1.3292772802524269) - rollout_corr/rollout_log_ppl:np.float64(0.28120700089493766) - rollout_corr/log_ppl_diff:np.float64(0.0010320210712961853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025203333934769034) - rollout_corr/log_ppl_diff_max:np.float64(0.011266171932220459) - rollout_corr/log_ppl_diff_min:np.float64(-0.00728076696395874) - rollout_corr/ppl_ratio:np.float64(1.0010370928794146) - rollout_corr/chi2_token:np.float64(0.002755629364401102) - rollout_corr/chi2_seq:np.float64(0.8209603114519268) - actor/pg_loss:np.float64(0.0054146344773471355) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001441872691430035) - actor/ppo_kl:np.float64(-3.021463783170475e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(149585.24241724727) - self_distillation/token_reweight_w_std:np.float64(2245182.637069527) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022315112873912) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10743028059368953) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.5845905467867851) - perf/mfu/actor:np.float64(0.031846668272466384) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.01116561889648) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008261048933491111 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008261048933491111 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean
(TaskRunner pid=2026458) Training Progress: 27%|██▋ | 27/100 [20:16<44:06, 36.25s/it]
(TaskRunner pid=2026458) step:28 - global_seqlen/min:15064 - global_seqlen/max:18912 - global_seqlen/minmax_diff:3848 - global_seqlen/balanced_min:17172 - global_seqlen/balanced_max:17302 - global_seqlen/mean:17191.25 - actor/entropy:0.26276373863220215 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491259455680847 - training/rollout_probs_diff_mean:0.005732021760195494 - training/rollout_probs_diff_std:0.014186778105795383 - training/rollout_actor_probs_pearson_corr:0.9981420636177063 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999222159385681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4750567970622797e-05 - rollout_corr/rollout_is_max:1.8682423830032349 - rollout_corr/rollout_is_min:0.4175392985343933 - rollout_corr/rollout_is_std:0.043118301779031754 - rollout_corr/rollout_is_eff_sample_size:0.9981441433187315 - rollout_corr/rollout_is_seq_mean:0.9998924732208252 - rollout_corr/rollout_is_seq_std:0.0026770406402647495 - rollout_corr/rollout_is_seq_max:1.0065722465515137 - rollout_corr/rollout_is_seq_min:0.9909303188323975 - rollout_corr/rollout_is_seq_max_deviation:0.009069681167602539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3176847058348358) - rollout_corr/training_log_ppl:np.float64(0.2727040337776998) - rollout_corr/kl:np.float64(0.0012852028201848498) - rollout_corr/k3_kl:np.float64(0.001238845826378565) - rollout_corr/rollout_ppl:np.float64(1.315975757315755) - rollout_corr/rollout_log_ppl:np.float64(0.27141882907017134) - rollout_corr/log_ppl_diff:np.float64(0.0012852047075284645) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026650355284800753) - rollout_corr/log_ppl_diff_max:np.float64(0.011441320180892944) - rollout_corr/log_ppl_diff_min:np.float64(-0.011876940727233887) - rollout_corr/ppl_ratio:np.float64(1.0012910023797303) - rollout_corr/chi2_token:np.float64(0.002501809736713767) - rollout_corr/chi2_seq:np.float64(1.4115001251921058) - actor/pg_loss:np.float64(0.005721935303881764) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001080054133581143) - actor/ppo_kl:np.float64(0.00020560735394870022) - actor/pg_clipfrac_lower:np.float64(3.0048076951061375e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(263679.60727620986) - self_distillation/token_reweight_w_std:np.float64(3103030.912776855) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007451251149178) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10346215419122018) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.6124091595411301) - perf/mfu/actor:np.float64(0.03192732787611963) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.95586776733398) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00606801500543952 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00606801500543952 - critic/returns/max:0.875 - critic/returns/min:-0.875 - respons
(TaskRunner pid=2026458) Training Progress: 28%|██▊ | 28/100 [20:47<41:35, 34.66s/it]
(TaskRunner pid=2026458) step:29 - global_seqlen/min:14445 - global_seqlen/max:22059 - global_seqlen/minmax_diff:7614 - global_seqlen/balanced_min:19100 - global_seqlen/balanced_max:20098 - global_seqlen/mean:19231.25 - actor/entropy:0.2399301379919052 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5590871572494507 - training/rollout_probs_diff_mean:0.005236997734755278 - training/rollout_probs_diff_std:0.014018451794981956 - training/rollout_actor_probs_pearson_corr:0.9981542229652405 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.34375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:1.0000776052474976 - rollout_corr/rollout_is_ratio_fraction_high:2.4095805201795883e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.819161040359177e-05 - rollout_corr/rollout_is_max:3.816781997680664 - rollout_corr/rollout_is_min:0.17573890089988708 - rollout_corr/rollout_is_std:0.04291044920682907 - rollout_corr/rollout_is_eff_sample_size:0.9981623150356519 - rollout_corr/rollout_is_seq_mean:0.9999245405197144 - rollout_corr/rollout_is_seq_std:0.0027032301295548677 - rollout_corr/rollout_is_seq_max:1.008720874786377 - rollout_corr/rollout_is_seq_min:0.992788553237915 - rollout_corr/rollout_is_seq_max_deviation:0.008720874786376953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3173197288997471) - rollout_corr/training_log_ppl:np.float64(0.27068548381794244) - rollout_corr/kl:np.float64(0.0009471329129642214) - rollout_corr/k3_kl:np.float64(0.001118057121630045) - rollout_corr/rollout_ppl:np.float64(1.3160036713816226) - rollout_corr/rollout_log_ppl:np.float64(0.2697383483755402) - rollout_corr/log_ppl_diff:np.float64(0.0009471354424022138) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024268297711387277) - rollout_corr/log_ppl_diff_max:np.float64(0.010676145553588867) - rollout_corr/log_ppl_diff_min:np.float64(-0.008322864770889282) - rollout_corr/ppl_ratio:np.float64(1.0009521523024887) - rollout_corr/chi2_token:np.float64(0.0025950868148356676) - rollout_corr/chi2_seq:np.float64(1.0168882303405553) - actor/pg_loss:np.float64(0.004738333867862821) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009898098714984371) - actor/ppo_kl:np.float64(-0.0001235502958607526) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.34375) - self_distillation/token_reweight_w_mean:np.float64(148981.53039407637) - self_distillation/token_reweight_w_std:np.float64(2396815.2922347095) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019751288928092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.088682441302808) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.5395381674170494) - perf/mfu/actor:np.float64(0.0350041185371077) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9663429260254) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.34375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014469530433416367 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014469530433416367 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/me
(TaskRunner pid=2026458) Training Progress: 29%|██▉ | 29/100 [21:24<41:59, 35.49s/it]
(TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30}
(TaskRunner pid=2026458) validation generation end
(TaskRunner pid=2026458) [prompt] system
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2026458) <reasoning>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) user
(TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2026458) Please reason step by step.
(TaskRunner pid=2026458) assistant
(TaskRunner pid=2026458) <think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) </think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) [response] <reasoning>
(TaskRunner pid=2026458) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the options provided. The product contains a key fragment: "Cc2ccc(Cl)cc2C(F)(F)F", which is a benzene ring with a chlorine substituent and a trifluoromethyl group. This fragment is likely derived from a reactant that contains a similar structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option A includes a structure with a benzene ring, a chlorine substituent, and a carboxylic acid group, but it does not contain the trifluoromethyl group.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option B includes a structure with a benzene ring and a chlorine substituent, but it lacks the trifluoromethyl group and has a different functional group arrangement.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option C includes a structure with a benzene ring, a chlorine substituent, and a trifluoromethyl group, and it matches the required fragment in the product. It also includes a carboxylic acid group, which is consistent with the structure of the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option D includes a structure with a benzene ring and a trifluoromethyl group, but it lacks the chlorine substituent and has a different functional group arrangement.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Therefore, Option C is the only correct reactant that matches the structure of the product.
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) C
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458) [ground_truth] A
(TaskRunner pid=2026458) [score] 0.0
(TaskRunner pid=2026458) [acc] 0.0
(TaskRunner pid=2026458) [pred] C
(TaskRunner pid=2026458) [incorrect_format] 1
(TaskRunner pid=2026458) [feedback]
(TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30
(WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor
(TaskRunner pid=2026458) step:30 - global_seqlen/min:15398 - global_seqlen/max:27317 - global_seqlen/minmax_diff:11919 - global_seqlen/balanced_min:18528 - global_seqlen/balanced_max:25696 - global_seqlen/mean:20322.875 - actor/entropy:0.21429221332073212 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29036805033683777 - training/rollout_probs_diff_mean:0.0045964838936924934 - training/rollout_probs_diff_std:0.012965062633156776 - training/rollout_actor_probs_pearson_corr:0.9982911944389343 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:0.9999114871025085 - rollout_corr/rollout_is_ratio_fraction_high:1.198825157189276e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.99412560404744e-05 - rollout_corr/rollout_is_max:2.0840299129486084 - rollout_corr/rollout_is_min:0.03206612169742584 - rollout_corr/rollout_is_std:0.04015055671334267 - rollout_corr/rollout_is_eff_sample_size:0.9983904085296447 - rollout_corr/rollout_is_seq_mean:0.9999212622642517 - rollout_corr/rollout_is_seq_std:0.003086200449615717 - rollout_corr/rollout_is_seq_max:1.0128618478775024 - rollout_corr/rollout_is_seq_min:0.9903483986854553 - rollout_corr/rollout_is_seq_max_deviation:0.012861847877502441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3321797889657319) - rollout_corr/training_log_ppl:np.float64(0.28194430993880815) - rollout_corr/kl:np.float64(0.0011635422050204625) - rollout_corr/k3_kl:np.float64(0.0012278429346110897) - rollout_corr/rollout_ppl:np.float64(1.3305623242631555) - rollout_corr/rollout_log_ppl:np.float64(0.2807807675008007) - rollout_corr/log_ppl_diff:np.float64(0.0011635424380074255) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028164471932541346) - rollout_corr/log_ppl_diff_max:np.float64(0.01170569658279419) - rollout_corr/log_ppl_diff_min:np.float64(-0.010286390781402588) - rollout_corr/ppl_ratio:np.float64(1.0011705257929862) - rollout_corr/chi2_token:np.float64(0.0025684768334031105) - rollout_corr/chi2_seq:np.float64(0.56981481006369) - actor/pg_loss:np.float64(0.004291482153348625) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010325129969714908) - actor/ppo_kl:np.float64(2.040402626590776e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(97715.03854049812) - self_distillation/token_reweight_w_std:np.float64(1362849.7468459418) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998477473855019) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08768687405972742) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.5216539651155472) - perf/mfu/actor:np.float64(0.03586380732978027) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9196662902832) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/reward/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1818003
(TaskRunner pid=2026458) Training Progress: 30%|███ | 30/100 [23:52<1:20:43, 69.19s/it]
(TaskRunner pid=2026458) step:31 - global_seqlen/min:15697 - global_seqlen/max:20497 - global_seqlen/minmax_diff:4800 - global_seqlen/balanced_min:18142 - global_seqlen/balanced_max:18440 - global_seqlen/mean:18179.75 - actor/entropy:0.24378487467765808 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4056262969970703 - training/rollout_probs_diff_mean:0.005305483937263489 - training/rollout_probs_diff_std:0.013869675807654858 - training/rollout_actor_probs_pearson_corr:0.9981986880302429 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0002444982528687 - rollout_corr/rollout_is_ratio_fraction_high:5.420200977823697e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.485351620242e-05 - rollout_corr/rollout_is_max:2.6529436111450195 - rollout_corr/rollout_is_min:0.2698281407356262 - rollout_corr/rollout_is_std:0.04314593970775604 - rollout_corr/rollout_is_eff_sample_size:0.9981427181140134 - rollout_corr/rollout_is_seq_mean:1.0003163814544678 - rollout_corr/rollout_is_seq_std:0.0029239451978355646 - rollout_corr/rollout_is_seq_max:1.0140212774276733 - rollout_corr/rollout_is_seq_min:0.9939373135566711 - rollout_corr/rollout_is_seq_max_deviation:0.01402127742767334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3019663002341986) - rollout_corr/training_log_ppl:np.float64(0.2597778052149806) - rollout_corr/kl:np.float64(0.001051263411028458) - rollout_corr/k3_kl:np.float64(0.0012008980110067569) - rollout_corr/rollout_ppl:np.float64(1.3005597270093858) - rollout_corr/rollout_log_ppl:np.float64(0.2587265418842435) - rollout_corr/log_ppl_diff:np.float64(0.0010512633307371289) - rollout_corr/log_ppl_abs_diff:np.float64(0.002555768267484382) - rollout_corr/log_ppl_diff_max:np.float64(0.010198384523391724) - rollout_corr/log_ppl_diff_min:np.float64(-0.010389238595962524) - rollout_corr/ppl_ratio:np.float64(1.00105693587102) - rollout_corr/chi2_token:np.float64(0.002714824862778187) - rollout_corr/chi2_seq:np.float64(1.5057584547903389) - actor/pg_loss:np.float64(0.004452571272850037) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007820382315912866) - actor/ppo_kl:np.float64(0.00035900160938950876) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(99177.65471042739) - self_distillation/token_reweight_w_std:np.float64(1529254.068720376) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9981596393045038) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09003465421847068) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.5145964473485947) - perf/mfu/actor:np.float64(0.0335625674385555) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.96116638183594) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012466462794691324 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012466462794691324 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:288.27343
(TaskRunner pid=2026458) Training Progress: 31%|███ | 31/100 [24:22<1:05:57, 57.36s/it]
(TaskRunner pid=2026458) step:32 - global_seqlen/min:13741 - global_seqlen/max:20093 - global_seqlen/minmax_diff:6352 - global_seqlen/balanced_min:17020 - global_seqlen/balanced_max:17022 - global_seqlen/mean:17021.5 - actor/entropy:0.2510213851928711 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3174631893634796 - training/rollout_probs_diff_mean:0.005626973696053028 - training/rollout_probs_diff_std:0.014290885999798775 - training/rollout_actor_probs_pearson_corr:0.9980513453483582 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0003660917282104 - rollout_corr/rollout_is_ratio_fraction_high:4.35514775745105e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.35514775745105e-05 - rollout_corr/rollout_is_max:2.588045358657837 - rollout_corr/rollout_is_min:0.49156951904296875 - rollout_corr/rollout_is_std:0.04363907128572464 - rollout_corr/rollout_is_eff_sample_size:0.9981006764057321 - rollout_corr/rollout_is_seq_mean:1.0004558563232422 - rollout_corr/rollout_is_seq_std:0.0026327052619308233 - rollout_corr/rollout_is_seq_max:1.0082557201385498 - rollout_corr/rollout_is_seq_min:0.9930630326271057 - rollout_corr/rollout_is_seq_max_deviation:0.008255720138549805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3111304240301251) - rollout_corr/training_log_ppl:np.float64(0.2656050497171236) - rollout_corr/kl:np.float64(0.0011632751664905072) - rollout_corr/k3_kl:np.float64(0.0012394887605751137) - rollout_corr/rollout_ppl:np.float64(1.3095909063704312) - rollout_corr/rollout_log_ppl:np.float64(0.2644417754927417) - rollout_corr/log_ppl_diff:np.float64(0.0011632742243818939) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027760078373830765) - rollout_corr/log_ppl_diff_max:np.float64(0.027847379446029663) - rollout_corr/log_ppl_diff_min:np.float64(-0.00688682496547699) - rollout_corr/ppl_ratio:np.float64(1.0011707202065736) - rollout_corr/chi2_token:np.float64(0.00264995195902884) - rollout_corr/chi2_seq:np.float64(1.4071137343998998) - actor/pg_loss:np.float64(0.003960146103054285) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013795204131383798) - actor/ppo_kl:np.float64(0.0005964390661219454) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(250012.3793359641) - self_distillation/token_reweight_w_std:np.float64(2860040.7311532954) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007816257420927) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07969121757196262) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.522880308330059) - perf/mfu/actor:np.float64(0.031496540810405246) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.58367156982422) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019834069535136223 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019834069535136223 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:269.0
(TaskRunner pid=2026458) Training Progress: 32%|███▏ | 32/100 [24:51<55:22, 48.86s/it]
(TaskRunner pid=2026458) step:33 - global_seqlen/min:15641 - global_seqlen/max:20620 - global_seqlen/minmax_diff:4979 - global_seqlen/balanced_min:18180 - global_seqlen/balanced_max:18182 - global_seqlen/mean:18181.0 - actor/entropy:0.28735455870628357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9959163069725037 - training/rollout_probs_diff_mean:0.00580747751519084 - training/rollout_probs_diff_std:0.015400327742099762 - training/rollout_actor_probs_pearson_corr:0.9980352520942688 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9997233152389526 - rollout_corr/rollout_is_ratio_fraction_high:1.423528101440752e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014235280104912817 - rollout_corr/rollout_is_max:2.051370859146118 - rollout_corr/rollout_is_min:0.0007158105727285147 - rollout_corr/rollout_is_std:0.04281798005104065 - rollout_corr/rollout_is_eff_sample_size:0.998169085060141 - rollout_corr/rollout_is_seq_mean:0.9997956156730652 - rollout_corr/rollout_is_seq_std:0.003073018742725253 - rollout_corr/rollout_is_seq_max:1.0114701986312866 - rollout_corr/rollout_is_seq_min:0.9903202056884766 - rollout_corr/rollout_is_seq_max_deviation:0.011470198631286621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3598110149614513) - rollout_corr/training_log_ppl:np.float64(0.30384603922721) - rollout_corr/kl:np.float64(0.0014454446327292203) - rollout_corr/k3_kl:np.float64(0.001423622587481077) - rollout_corr/rollout_ppl:np.float64(1.3577961758710444) - rollout_corr/rollout_log_ppl:np.float64(0.3024005940533243) - rollout_corr/log_ppl_diff:np.float64(0.0014454451738856733) - rollout_corr/log_ppl_abs_diff:np.float64(0.002951021713670343) - rollout_corr/log_ppl_diff_max:np.float64(0.02602332830429077) - rollout_corr/log_ppl_diff_min:np.float64(-0.010346651077270508) - rollout_corr/ppl_ratio:np.float64(1.001454961951822) - rollout_corr/chi2_token:np.float64(0.002324617700651288) - rollout_corr/chi2_seq:np.float64(0.6557224565185606) - actor/pg_loss:np.float64(0.0037985126255080104) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009114227987083723) - actor/ppo_kl:np.float64(7.80257138011109e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(186856.52135437424) - self_distillation/token_reweight_w_std:np.float64(2469605.76739274) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999388714786619) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09890055964933708) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5402719005942345) - perf/mfu/actor:np.float64(0.03298787544887611) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.31770038604736) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003960967063903809 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003960967063903809 - critic/returns/max:0.875 - critic/returns/min:-0.875 - respon
(TaskRunner pid=2026458) Training Progress: 33%|███▎ | 33/100 [25:20<48:01, 43.01s/it]
(TaskRunner pid=2026458) step:34 - global_seqlen/min:15048 - global_seqlen/max:22716 - global_seqlen/minmax_diff:7668 - global_seqlen/balanced_min:18406 - global_seqlen/balanced_max:18407 - global_seqlen/mean:18406.75 - actor/entropy:0.2822253406047821 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3892279863357544 - training/rollout_probs_diff_mean:0.00599341094493866 - training/rollout_probs_diff_std:0.014615834690630436 - training/rollout_actor_probs_pearson_corr:0.9981156587600708 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9999809265136719 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00012492018868215382 - rollout_corr/rollout_is_max:1.9517436027526855 - rollout_corr/rollout_is_min:0.10854519158601761 - rollout_corr/rollout_is_std:0.04540359228849411 - rollout_corr/rollout_is_eff_sample_size:0.9979426360193112 - rollout_corr/rollout_is_seq_mean:0.9999065399169922 - rollout_corr/rollout_is_seq_std:0.003091080579906702 - rollout_corr/rollout_is_seq_max:1.0100407600402832 - rollout_corr/rollout_is_seq_min:0.9879905581474304 - rollout_corr/rollout_is_seq_max_deviation:0.01200944185256958 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3448055190965533) - rollout_corr/training_log_ppl:np.float64(0.29254834007588215) - rollout_corr/kl:np.float64(0.001122740535322464) - rollout_corr/k3_kl:np.float64(0.0011786174198959998) - rollout_corr/rollout_ppl:np.float64(1.3432867326773703) - rollout_corr/rollout_log_ppl:np.float64(0.2914255977375433) - rollout_corr/log_ppl_diff:np.float64(0.0011227423383388668) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025718869001138955) - rollout_corr/log_ppl_diff_max:np.float64(0.014177501201629639) - rollout_corr/log_ppl_diff_min:np.float64(-0.006468132138252258) - rollout_corr/ppl_ratio:np.float64(1.0011283229105175) - rollout_corr/chi2_token:np.float64(0.002466450445353985) - rollout_corr/chi2_seq:np.float64(0.9479083714541048) - actor/pg_loss:np.float64(0.005841488018631935) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011660458308142552) - actor/ppo_kl:np.float64(-8.996075499190237e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(205396.02536904486) - self_distillation/token_reweight_w_std:np.float64(2610298.4065771694) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991585344541818) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12581025002873503) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.6120489835739136) - perf/mfu/actor:np.float64(0.03423697654860308) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.21736526489258) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012471199035644531 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012471199035644531 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:281.
(TaskRunner pid=2026458) Training Progress: 34%|███▍ | 34/100 [25:50<43:03, 39.14s/it]
(TaskRunner pid=2026458) step:35 - global_seqlen/min:14987 - global_seqlen/max:19906 - global_seqlen/minmax_diff:4919 - global_seqlen/balanced_min:16938 - global_seqlen/balanced_max:16939 - global_seqlen/mean:16938.25 - actor/entropy:0.294697642326355 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22079068422317505 - training/rollout_probs_diff_mean:0.005864571314305067 - training/rollout_probs_diff_std:0.013605127111077309 - training/rollout_actor_probs_pearson_corr:0.9984296560287476 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:1.00005042552948 - rollout_corr/rollout_is_ratio_fraction_high:3.1109037081478164e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.1109037081478164e-05 - rollout_corr/rollout_is_max:2.7191102504730225 - rollout_corr/rollout_is_min:0.37482571601867676 - rollout_corr/rollout_is_std:0.04332791268825531 - rollout_corr/rollout_is_eff_sample_size:0.9981263285522803 - rollout_corr/rollout_is_seq_mean:1.0001163482666016 - rollout_corr/rollout_is_seq_std:0.0030649646650999784 - rollout_corr/rollout_is_seq_max:1.0084450244903564 - rollout_corr/rollout_is_seq_min:0.989280641078949 - rollout_corr/rollout_is_seq_max_deviation:0.010719358921051025 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3609930002130568) - rollout_corr/training_log_ppl:np.float64(0.3051015918608755) - rollout_corr/kl:np.float64(0.0011746971918284999) - rollout_corr/k3_kl:np.float64(0.0011032617354658214) - rollout_corr/rollout_ppl:np.float64(1.3593582352623343) - rollout_corr/rollout_log_ppl:np.float64(0.30392689420841634) - rollout_corr/log_ppl_diff:np.float64(0.0011746976524591446) - rollout_corr/log_ppl_abs_diff:np.float64(0.002850949880667031) - rollout_corr/log_ppl_diff_max:np.float64(0.012322753667831421) - rollout_corr/log_ppl_diff_min:np.float64(-0.010998740792274475) - rollout_corr/ppl_ratio:np.float64(1.0011811451986432) - rollout_corr/chi2_token:np.float64(0.0021009566262364388) - rollout_corr/chi2_seq:np.float64(1.5435644236858934) - actor/pg_loss:np.float64(0.0014815301401540637) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020538957323879004) - actor/ppo_kl:np.float64(0.00030627488916046275) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(94700.63521376462) - self_distillation/token_reweight_w_std:np.float64(1075744.331360741) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004025450907648) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03615615499438718) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.3742603175342083) - perf/mfu/actor:np.float64(0.031170659142369998) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1671371459961) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029106393922120333 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0029106393922120333 - critic/returns/max:0.875 - critic/returns/min:-0.75 - respo
(TaskRunner pid=2026458) Training Progress: 35%|███▌ | 35/100 [26:19<38:59, 35.99s/it]
(TaskRunner pid=2026458) step:36 - global_seqlen/min:14431 - global_seqlen/max:20659 - global_seqlen/minmax_diff:6228 - global_seqlen/balanced_min:17408 - global_seqlen/balanced_max:17413 - global_seqlen/mean:17411.75 - actor/entropy:0.2818928360939026 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30500367283821106 - training/rollout_probs_diff_mean:0.005986991338431835 - training/rollout_probs_diff_std:0.014329608529806137 - training/rollout_actor_probs_pearson_corr:0.9982293248176575 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0000543594360352 - rollout_corr/rollout_is_ratio_fraction_high:1.4678683328384068e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.871473331353627e-05 - rollout_corr/rollout_is_max:3.0542242527008057 - rollout_corr/rollout_is_min:0.18286029994487762 - rollout_corr/rollout_is_std:0.044746387749910355 - rollout_corr/rollout_is_eff_sample_size:0.9980018804571298 - rollout_corr/rollout_is_seq_mean:1.0000989437103271 - rollout_corr/rollout_is_seq_std:0.0029557279776781797 - rollout_corr/rollout_is_seq_max:1.0128530263900757 - rollout_corr/rollout_is_seq_min:0.992638111114502 - rollout_corr/rollout_is_seq_max_deviation:0.012853026390075684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3384348950348794) - rollout_corr/training_log_ppl:np.float64(0.28769120943616144) - rollout_corr/kl:np.float64(0.0010638975770804748) - rollout_corr/k3_kl:np.float64(0.0012365762453327989) - rollout_corr/rollout_ppl:np.float64(1.3369859019294381) - rollout_corr/rollout_log_ppl:np.float64(0.28662730925134383) - rollout_corr/log_ppl_diff:np.float64(0.0010639001848176122) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025838720030151308) - rollout_corr/log_ppl_diff_max:np.float64(0.01416441798210144) - rollout_corr/log_ppl_diff_min:np.float64(-0.009993910789489746) - rollout_corr/ppl_ratio:np.float64(1.0010696118697524) - rollout_corr/chi2_token:np.float64(0.002987496554851532) - rollout_corr/chi2_seq:np.float64(0.8903114418499172) - actor/pg_loss:np.float64(0.0051360129145905375) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000932457276576315) - actor/ppo_kl:np.float64(0.00010139281163645819) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(95758.95235729823) - self_distillation/token_reweight_w_std:np.float64(1392223.179759777) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9973206941504031) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10905869025737047) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.5497298240661621) - perf/mfu/actor:np.float64(0.03256102036135432) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1156120300293) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007596218958497047 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007596218958497047 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_le
(TaskRunner pid=2026458) Training Progress: 36%|███▌ | 36/100 [26:48<36:12, 33.94s/it]
(TaskRunner pid=2026458) step:37 - global_seqlen/min:15833 - global_seqlen/max:21178 - global_seqlen/minmax_diff:5345 - global_seqlen/balanced_min:18583 - global_seqlen/balanced_max:18587 - global_seqlen/mean:18585.625 - actor/entropy:0.285804808139801 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4652300179004669 - training/rollout_probs_diff_mean:0.005965293850749731 - training/rollout_probs_diff_std:0.014233550056815147 - training/rollout_actor_probs_pearson_corr:0.9982559680938721 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:1.0002163648605347 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.9148995963623747e-05 - rollout_corr/rollout_is_max:1.695287823677063 - rollout_corr/rollout_is_min:0.16980920732021332 - rollout_corr/rollout_is_std:0.04377271980047226 - rollout_corr/rollout_is_eff_sample_size:0.9980885633504123 - rollout_corr/rollout_is_seq_mean:1.000288486480713 - rollout_corr/rollout_is_seq_std:0.0028211281169205904 - rollout_corr/rollout_is_seq_max:1.0087376832962036 - rollout_corr/rollout_is_seq_min:0.9919080138206482 - rollout_corr/rollout_is_seq_max_deviation:0.008737683296203613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3575056260451674) - rollout_corr/training_log_ppl:np.float64(0.3016296330606565) - rollout_corr/kl:np.float64(0.0007570147956474926) - rollout_corr/k3_kl:np.float64(0.0012164926273499077) - rollout_corr/rollout_ppl:np.float64(1.3564373790286481) - rollout_corr/rollout_log_ppl:np.float64(0.30087261548032984) - rollout_corr/log_ppl_diff:np.float64(0.0007570175803266466) - rollout_corr/log_ppl_abs_diff:np.float64(0.002560925087891519) - rollout_corr/log_ppl_diff_max:np.float64(0.011357903480529785) - rollout_corr/log_ppl_diff_min:np.float64(-0.008871495723724365) - rollout_corr/ppl_ratio:np.float64(1.000762677518651) - rollout_corr/chi2_token:np.float64(0.0033016859088093042) - rollout_corr/chi2_seq:np.float64(1.0105145464185625) - actor/pg_loss:np.float64(0.0031967939576134086) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006711441328661749) - actor/ppo_kl:np.float64(3.3348735911431504e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(214368.9940047427) - self_distillation/token_reweight_w_std:np.float64(2919033.6983316364) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007499558851123) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07807663449784741) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.4402663931250572) - perf/mfu/actor:np.float64(0.034623907048585915) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.02531814575195) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031590224243700504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031590224243700504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mea
(TaskRunner pid=2026458) Training Progress: 37%|███▋ | 37/100 [27:18<34:12, 32.58s/it]
(TaskRunner pid=2026458) step:38 - global_seqlen/min:15292 - global_seqlen/max:19759 - global_seqlen/minmax_diff:4467 - global_seqlen/balanced_min:17036 - global_seqlen/balanced_max:17040 - global_seqlen/mean:17038.625 - actor/entropy:0.2831912040710449 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4492228925228119 - training/rollout_probs_diff_mean:0.006005707662552595 - training/rollout_probs_diff_std:0.014447803609073162 - training/rollout_actor_probs_pearson_corr:0.9981924891471863 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9998399019241333 - rollout_corr/rollout_is_ratio_fraction_high:2.9010312573518604e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3515468860277906e-05 - rollout_corr/rollout_is_max:4.976550579071045 - rollout_corr/rollout_is_min:0.1723451316356659 - rollout_corr/rollout_is_std:0.045169975608587265 - rollout_corr/rollout_is_eff_sample_size:0.997963056153362 - rollout_corr/rollout_is_seq_mean:0.999801754951477 - rollout_corr/rollout_is_seq_std:0.0029568150639533997 - rollout_corr/rollout_is_seq_max:1.01389741897583 - rollout_corr/rollout_is_seq_min:0.9917296171188354 - rollout_corr/rollout_is_seq_max_deviation:0.013897418975830078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3477132506668568) - rollout_corr/training_log_ppl:np.float64(0.29560438107000664) - rollout_corr/kl:np.float64(0.0015795158470695014) - rollout_corr/k3_kl:np.float64(0.0011915537743334426) - rollout_corr/rollout_ppl:np.float64(1.3455825243145227) - rollout_corr/rollout_log_ppl:np.float64(0.29402486264007166) - rollout_corr/log_ppl_diff:np.float64(0.0015795184299349785) - rollout_corr/log_ppl_abs_diff:np.float64(0.002752322005107999) - rollout_corr/log_ppl_diff_max:np.float64(0.010404467582702637) - rollout_corr/log_ppl_diff_min:np.float64(-0.006978675723075867) - rollout_corr/ppl_ratio:np.float64(1.0015854325611144) - rollout_corr/chi2_token:np.float64(0.0016510849818587303) - rollout_corr/chi2_seq:np.float64(0.42648625490255654) - actor/pg_loss:np.float64(0.004350315313786268) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001112287538489909) - actor/ppo_kl:np.float64(0.0002975818272203945) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(277881.75766049954) - self_distillation/token_reweight_w_std:np.float64(3405953.521727615) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997533587738872) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09852648590458557) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.5438188463449478) - perf/mfu/actor:np.float64(0.03193194428398073) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.85803985595703) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00237884558737278 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00237884558737278 - critic/returns/max:0.875 - critic/returns/min:-0.875
(TaskRunner pid=2026458) Training Progress: 38%|███▊ | 38/100 [27:46<32:25, 31.38s/it]
(TaskRunner pid=2026458) step:39 - global_seqlen/min:15017 - global_seqlen/max:21775 - global_seqlen/minmax_diff:6758 - global_seqlen/balanced_min:17748 - global_seqlen/balanced_max:17751 - global_seqlen/mean:17750.375 - actor/entropy:0.2934376001358032 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744115471839905 - training/rollout_probs_diff_mean:0.006076316349208355 - training/rollout_probs_diff_std:0.014462748542428017 - training/rollout_actor_probs_pearson_corr:0.9982326030731201 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001943111419678 - rollout_corr/rollout_is_ratio_fraction_high:2.8678357921307907e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.735671584261581e-05 - rollout_corr/rollout_is_max:2.3144755363464355 - rollout_corr/rollout_is_min:0.44314810633659363 - rollout_corr/rollout_is_std:0.04461565613746643 - rollout_corr/rollout_is_eff_sample_size:0.9980141101447896 - rollout_corr/rollout_is_seq_mean:1.0003681182861328 - rollout_corr/rollout_is_seq_std:0.0030234723817557096 - rollout_corr/rollout_is_seq_max:1.012935996055603 - rollout_corr/rollout_is_seq_min:0.991753876209259 - rollout_corr/rollout_is_seq_max_deviation:0.012935996055603027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3561590211465955) - rollout_corr/training_log_ppl:np.float64(0.30050787163781933) - rollout_corr/kl:np.float64(0.0008042539968293738) - rollout_corr/k3_kl:np.float64(0.0011359995904740572) - rollout_corr/rollout_ppl:np.float64(1.355069079902023) - rollout_corr/rollout_log_ppl:np.float64(0.2997036176966503) - rollout_corr/log_ppl_diff:np.float64(0.0008042539411690086) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024447946052532643) - rollout_corr/log_ppl_diff_max:np.float64(0.010187238454818726) - rollout_corr/log_ppl_diff_min:np.float64(-0.010493099689483643) - rollout_corr/ppl_ratio:np.float64(1.0008093006908894) - rollout_corr/chi2_token:np.float64(0.0029845864046365023) - rollout_corr/chi2_seq:np.float64(0.9699735399335623) - actor/pg_loss:np.float64(0.003664600197225809) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006868128334645007) - actor/ppo_kl:np.float64(0.00012619273251779362) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(231536.99509470444) - self_distillation/token_reweight_w_std:np.float64(3103805.8869296443) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012708925642073) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08035396633204073) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.45319733768701553) - perf/mfu/actor:np.float64(0.03298738163573484) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.00569915771484) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005079654045403004 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005079654045403004 - critic/returns/max:0.875 - critic/returns/min:-0.875 -
(TaskRunner pid=2026458) Training Progress: 39%|███▉ | 39/100 [28:15<31:09, 30.65s/it]
(TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40}
(TaskRunner pid=2026458) validation generation end
(TaskRunner pid=2026458) [prompt] system
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2026458) <reasoning>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) user
(TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2026458) Please reason step by step.
(TaskRunner pid=2026458) assistant
(TaskRunner pid=2026458) <think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) </think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) [response] <reasoning>
(TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the options provided. The product contains a nitrogen atom bonded to a carbon chain, with a substituted aromatic ring (Cc2ccc(Cl)cc2) and a trifluoromethyl group (C(F)(F)F).
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option A includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. The nitrogen is bonded to a substituted aromatic ring with a chlorine atom and a trifluoromethyl group, matching the structure of the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option B includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it lacks the specific chlorine and trifluoromethyl groups.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option C includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it includes an oxygen group (Oc2) instead of the chlorine and trifluoromethyl groups.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option D includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it includes a double bond (CC=C) and a different trifluoromethyl group.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Based on the structural analysis, Option A is the only one that matches the product's structure, making it the correct reactant.
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458) [ground_truth] A
(TaskRunner pid=2026458) [score] 1.0
(TaskRunner pid=2026458) [acc] 1.0
(TaskRunner pid=2026458) [pred] A
(TaskRunner pid=2026458) [incorrect_format] 1
(TaskRunner pid=2026458) [feedback]
(TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40
(WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor
(TaskRunner pid=2026458) step:40 - global_seqlen/min:16589 - global_seqlen/max:23076 - global_seqlen/minmax_diff:6487 - global_seqlen/balanced_min:19038 - global_seqlen/balanced_max:19042 - global_seqlen/mean:19040.5 - actor/entropy:0.3083128035068512 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48448076844215393 - training/rollout_probs_diff_mean:0.006092434749007225 - training/rollout_probs_diff_std:0.01408663671463728 - training/rollout_actor_probs_pearson_corr:0.9983260631561279 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0004887580871582 - rollout_corr/rollout_is_ratio_fraction_high:2.767170371953398e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.383585185976699e-05 - rollout_corr/rollout_is_max:2.5501346588134766 - rollout_corr/rollout_is_min:0.2322635054588318 - rollout_corr/rollout_is_std:0.044838204979896545 - rollout_corr/rollout_is_eff_sample_size:0.9979954688959803 - rollout_corr/rollout_is_seq_mean:1.0004291534423828 - rollout_corr/rollout_is_seq_std:0.00280501926317811 - rollout_corr/rollout_is_seq_max:1.0100523233413696 - rollout_corr/rollout_is_seq_min:0.9925862550735474 - rollout_corr/rollout_is_seq_max_deviation:0.010052323341369629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3748237513937056) - rollout_corr/training_log_ppl:np.float64(0.3138403666380327) - rollout_corr/kl:np.float64(0.0008364543262393553) - rollout_corr/k3_kl:np.float64(0.0011391592736345046) - rollout_corr/rollout_ppl:np.float64(1.3736085868440568) - rollout_corr/rollout_log_ppl:np.float64(0.3130039106181357) - rollout_corr/log_ppl_diff:np.float64(0.0008364560198970139) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023778348113410175) - rollout_corr/log_ppl_diff_max:np.float64(0.009519308805465698) - rollout_corr/log_ppl_diff_min:np.float64(-0.0073702335357666016) - rollout_corr/ppl_ratio:np.float64(1.0008412639144808) - rollout_corr/chi2_token:np.float64(0.0029417627956718206) - rollout_corr/chi2_seq:np.float64(0.8329082692507654) - actor/pg_loss:np.float64(0.0021506232442334294) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047293603620346403) - actor/ppo_kl:np.float64(0.0002316894797989022) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(45010.86430421425) - self_distillation/token_reweight_w_std:np.float64(529273.9801185789) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998293393291533) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05143398354994133) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.38673160597682) - perf/mfu/actor:np.float64(0.03526754309862706) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.86711502075195) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/reward/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1521073
(TaskRunner pid=2026458) Training Progress: 40%|████ | 40/100 [30:44<1:06:02, 66.04s/it]
(TaskRunner pid=2026458) step:41 - global_seqlen/min:14161 - global_seqlen/max:22430 - global_seqlen/minmax_diff:8269 - global_seqlen/balanced_min:17529 - global_seqlen/balanced_max:17569 - global_seqlen/mean:17540.0 - actor/entropy:0.28421929478645325 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3167548179626465 - training/rollout_probs_diff_mean:0.005681131035089493 - training/rollout_probs_diff_std:0.013473561964929104 - training/rollout_actor_probs_pearson_corr:0.9984502792358398 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0002758502960205 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.967482328414917 - rollout_corr/rollout_is_min:0.5226783752441406 - rollout_corr/rollout_is_std:0.04252113774418831 - rollout_corr/rollout_is_eff_sample_size:0.998196166076475 - rollout_corr/rollout_is_seq_mean:1.000309944152832 - rollout_corr/rollout_is_seq_std:0.002816120395436883 - rollout_corr/rollout_is_seq_max:1.0100390911102295 - rollout_corr/rollout_is_seq_min:0.9916365146636963 - rollout_corr/rollout_is_seq_max_deviation:0.010039091110229492 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3460121774114668) - rollout_corr/training_log_ppl:np.float64(0.29434393605333753) - rollout_corr/kl:np.float64(0.0006344841518632904) - rollout_corr/k3_kl:np.float64(0.0010341189710061371) - rollout_corr/rollout_ppl:np.float64(1.3451492232270539) - rollout_corr/rollout_log_ppl:np.float64(0.2937094499648083) - rollout_corr/log_ppl_diff:np.float64(0.0006344860885292292) - rollout_corr/log_ppl_abs_diff:np.float64(0.002157465787604451) - rollout_corr/log_ppl_diff_max:np.float64(0.00896558165550232) - rollout_corr/log_ppl_diff_min:np.float64(-0.008629709482192993) - rollout_corr/ppl_ratio:np.float64(1.000638237921521) - rollout_corr/chi2_token:np.float64(0.0028806456830352545) - rollout_corr/chi2_seq:np.float64(0.6779269557446241) - actor/pg_loss:np.float64(0.004367844783701003) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004992729859623068) - actor/ppo_kl:np.float64(5.57020750591164e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(156417.83956303308) - self_distillation/token_reweight_w_std:np.float64(2357416.5479677515) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000363286351785) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0830252695013769) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.500577874481678) - perf/mfu/actor:np.float64(0.032587394156940674) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.82418060302734) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022402419708669186 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022402419708669186 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:284.21875 - response_length/max:788.0 - response_length/min:110.0
(TaskRunner pid=2026458) Training Progress: 41%|████ | 41/100 [31:14<54:26, 55.37s/it]
(TaskRunner pid=2026458) step:42 - global_seqlen/min:15278 - global_seqlen/max:20664 - global_seqlen/minmax_diff:5386 - global_seqlen/balanced_min:18018 - global_seqlen/balanced_max:18114 - global_seqlen/mean:18031.0 - actor/entropy:0.2842845022678375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.289776086807251 - training/rollout_probs_diff_mean:0.005445326212793589 - training/rollout_probs_diff_std:0.013068844564259052 - training/rollout_actor_probs_pearson_corr:0.9985550045967102 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9997981786727905 - rollout_corr/rollout_is_ratio_fraction_high:1.2730420166917611e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.5460840333835222e-05 - rollout_corr/rollout_is_max:2.036485433578491 - rollout_corr/rollout_is_min:0.43032172322273254 - rollout_corr/rollout_is_std:0.04114258661866188 - rollout_corr/rollout_is_eff_sample_size:0.998309494689457 - rollout_corr/rollout_is_seq_mean:0.999789297580719 - rollout_corr/rollout_is_seq_std:0.0025704929139465094 - rollout_corr/rollout_is_seq_max:1.008382797241211 - rollout_corr/rollout_is_seq_min:0.9924810528755188 - rollout_corr/rollout_is_seq_max_deviation:0.008382797241210938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3495313432067633) - rollout_corr/training_log_ppl:np.float64(0.2967612352222204) - rollout_corr/kl:np.float64(0.0012985346789022856) - rollout_corr/k3_kl:np.float64(0.0010939320847001) - rollout_corr/rollout_ppl:np.float64(1.3477259748615324) - rollout_corr/rollout_log_ppl:np.float64(0.29546269873389974) - rollout_corr/log_ppl_diff:np.float64(0.0012985364883206785) - rollout_corr/log_ppl_abs_diff:np.float64(0.002426193736027926) - rollout_corr/log_ppl_diff_max:np.float64(0.00859573483467102) - rollout_corr/log_ppl_diff_min:np.float64(-0.006498366594314575) - rollout_corr/ppl_ratio:np.float64(1.0013033242430538) - rollout_corr/chi2_token:np.float64(0.001749418443068862) - rollout_corr/chi2_seq:np.float64(0.668122704140842) - actor/pg_loss:np.float64(0.0077214669436216354) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001590086207670538) - actor/ppo_kl:np.float64(0.0002024844293835315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(176662.73121298593) - self_distillation/token_reweight_w_std:np.float64(2621435.3882032083) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9981021112762392) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14360627194400877) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.6128042787313461) - perf/mfu/actor:np.float64(0.03360615388418303) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.27114486694336) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003445170121267438 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003445170121267438 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_
(TaskRunner pid=2026458) Training Progress: 42%|████▏ | 42/100 [31:46<46:36, 48.21s/it]
(TaskRunner pid=2026458) step:43 - global_seqlen/min:17746 - global_seqlen/max:21005 - global_seqlen/minmax_diff:3259 - global_seqlen/balanced_min:19023 - global_seqlen/balanced_max:19027 - global_seqlen/mean:19025.375 - actor/entropy:0.29123467206954956 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.277299165725708 - training/rollout_probs_diff_mean:0.0057848598808050156 - training/rollout_probs_diff_std:0.013805463910102844 - training/rollout_actor_probs_pearson_corr:0.9984029531478882 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9998417496681213 - rollout_corr/rollout_is_ratio_fraction_high:1.3395668247540016e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.697834032820538e-05 - rollout_corr/rollout_is_max:2.0824544429779053 - rollout_corr/rollout_is_min:0.3074188232421875 - rollout_corr/rollout_is_std:0.0433402918279171 - rollout_corr/rollout_is_eff_sample_size:0.9981245471106103 - rollout_corr/rollout_is_seq_mean:0.9999098181724548 - rollout_corr/rollout_is_seq_std:0.0025561333168298006 - rollout_corr/rollout_is_seq_max:1.0072154998779297 - rollout_corr/rollout_is_seq_min:0.9910159111022949 - rollout_corr/rollout_is_seq_max_deviation:0.008984088897705078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3661411092616618) - rollout_corr/training_log_ppl:np.float64(0.30844287818763405) - rollout_corr/kl:np.float64(0.001555773359804391) - rollout_corr/k3_kl:np.float64(0.0014595502659631165) - rollout_corr/rollout_ppl:np.float64(1.3639800301752985) - rollout_corr/rollout_log_ppl:np.float64(0.30688710472895764) - rollout_corr/log_ppl_diff:np.float64(0.0015557734586764127) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026391229184810072) - rollout_corr/log_ppl_diff_max:np.float64(0.011147260665893555) - rollout_corr/log_ppl_diff_min:np.float64(-0.006583303213119507) - rollout_corr/ppl_ratio:np.float64(1.0015615709125996) - rollout_corr/chi2_token:np.float64(0.0026983728166669607) - rollout_corr/chi2_seq:np.float64(0.5271497289650142) - actor/pg_loss:np.float64(0.0038334736600518227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013588511892521637) - actor/ppo_kl:np.float64(0.00048035153239256445) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(120698.68936810177) - self_distillation/token_reweight_w_std:np.float64(1773706.5877801427) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000403418438509) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07702912960667163) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.46645358949899673) - perf/mfu/actor:np.float64(0.03541705134193752) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.77776336669922) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00016074800805654377 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00016074800805654377 - critic/returns/max:0.875 - critic/returns/m
(TaskRunner pid=2026458) Training Progress: 43%|████▎ | 43/100 [32:15<40:30, 42.64s/it]
(TaskRunner pid=2026458) step:44 - global_seqlen/min:16682 - global_seqlen/max:21465 - global_seqlen/minmax_diff:4783 - global_seqlen/balanced_min:19287 - global_seqlen/balanced_max:19289 - global_seqlen/mean:19287.75 - actor/entropy:0.2704760432243347 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26803696155548096 - training/rollout_probs_diff_mean:0.005588939879089594 - training/rollout_probs_diff_std:0.013731644488871098 - training/rollout_actor_probs_pearson_corr:0.9983340501785278 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000927448272705 - rollout_corr/rollout_is_ratio_fraction_high:2.68305120698642e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.049153257161379e-05 - rollout_corr/rollout_is_max:2.0968687534332275 - rollout_corr/rollout_is_min:0.22089464962482452 - rollout_corr/rollout_is_std:0.04294377565383911 - rollout_corr/rollout_is_eff_sample_size:0.9981595832973885 - rollout_corr/rollout_is_seq_mean:1.0000972747802734 - rollout_corr/rollout_is_seq_std:0.00284771085716784 - rollout_corr/rollout_is_seq_max:1.0088731050491333 - rollout_corr/rollout_is_seq_min:0.9906178116798401 - rollout_corr/rollout_is_seq_max_deviation:0.009382188320159912 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3369123949669302) - rollout_corr/training_log_ppl:np.float64(0.2868449309025891) - rollout_corr/kl:np.float64(0.0009355866875218055) - rollout_corr/k3_kl:np.float64(0.0010745994904937106) - rollout_corr/rollout_ppl:np.float64(1.335645436309278) - rollout_corr/rollout_log_ppl:np.float64(0.28590934313251637) - rollout_corr/log_ppl_diff:np.float64(0.0009355877700727433) - rollout_corr/log_ppl_abs_diff:np.float64(0.002383651997661218) - rollout_corr/log_ppl_diff_max:np.float64(0.011576682329177856) - rollout_corr/log_ppl_diff_min:np.float64(-0.006330162286758423) - rollout_corr/ppl_ratio:np.float64(1.00094054476358) - rollout_corr/chi2_token:np.float64(0.00241070962511003) - rollout_corr/chi2_seq:np.float64(0.8048715838231146) - actor/pg_loss:np.float64(0.0025291290367022157) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006139230463304557) - actor/ppo_kl:np.float64(5.7461318307527876e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(186553.62624621042) - self_distillation/token_reweight_w_std:np.float64(2401896.473150722) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013129995204508) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05587577508413233) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.40642886608839035) - perf/mfu/actor:np.float64(0.03559055533852201) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.5995101928711) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029178180266171694 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0029178180266171694 - critic/returns/max:0.875 - critic/returns/min:-0.875 - resp
(TaskRunner pid=2026458) Training Progress: 44%|████▍ | 44/100 [32:46<36:19, 38.91s/it]
(TaskRunner pid=2026458) step:45 - global_seqlen/min:15729 - global_seqlen/max:24872 - global_seqlen/minmax_diff:9143 - global_seqlen/balanced_min:18453 - global_seqlen/balanced_max:18459 - global_seqlen/mean:18456.25 - actor/entropy:0.28802311420440674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29810237884521484 - training/rollout_probs_diff_mean:0.005760597065091133 - training/rollout_probs_diff_std:0.013836598955094814 - training/rollout_actor_probs_pearson_corr:0.9983675479888916 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:1.000093698501587 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.471605279832147e-05 - rollout_corr/rollout_is_max:1.7223904132843018 - rollout_corr/rollout_is_min:0.3712316155433655 - rollout_corr/rollout_is_std:0.04281449690461159 - rollout_corr/rollout_is_eff_sample_size:0.9981707478872216 - rollout_corr/rollout_is_seq_mean:1.000179648399353 - rollout_corr/rollout_is_seq_std:0.002977126045152545 - rollout_corr/rollout_is_seq_max:1.008954644203186 - rollout_corr/rollout_is_seq_min:0.9924071431159973 - rollout_corr/rollout_is_seq_max_deviation:0.008954644203186035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3517238958738744) - rollout_corr/training_log_ppl:np.float64(0.2982309421640821) - rollout_corr/kl:np.float64(0.0008551144521504739) - rollout_corr/k3_kl:np.float64(0.0011088182236562716) - rollout_corr/rollout_ppl:np.float64(1.350550597999245) - rollout_corr/rollout_log_ppl:np.float64(0.29737582663074136) - rollout_corr/log_ppl_diff:np.float64(0.0008551155333407223) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025770837091840804) - rollout_corr/log_ppl_diff_max:np.float64(0.009607642889022827) - rollout_corr/log_ppl_diff_min:np.float64(-0.007143139839172363) - rollout_corr/ppl_ratio:np.float64(1.000860691536218) - rollout_corr/chi2_token:np.float64(0.002733499277383089) - rollout_corr/chi2_seq:np.float64(0.8130777420010418) - actor/pg_loss:np.float64(0.002860508277080953) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007193945029939641) - actor/ppo_kl:np.float64(8.01470097968604e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(108369.78586033895) - self_distillation/token_reweight_w_std:np.float64(1441947.6739233742) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9984894776716828) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07410955597879365) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.417113721370697) - perf/mfu/actor:np.float64(0.03440527561058198) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.48589324951172) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00041734983096830547 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00041734983096830547 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean
(TaskRunner pid=2026458) Training Progress: 45%|████▌ | 45/100 [33:15<33:03, 36.06s/it]
(TaskRunner pid=2026458) step:46 - global_seqlen/min:16485 - global_seqlen/max:20444 - global_seqlen/minmax_diff:3959 - global_seqlen/balanced_min:17819 - global_seqlen/balanced_max:17823 - global_seqlen/mean:17821.125 - actor/entropy:0.29167261719703674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032017827033997 - training/rollout_probs_diff_mean:0.005424427799880505 - training/rollout_probs_diff_std:0.013135196641087532 - training/rollout_actor_probs_pearson_corr:0.9985377788543701 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9998041391372681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7731173759093508e-05 - rollout_corr/rollout_is_max:1.7920777797698975 - rollout_corr/rollout_is_min:0.19150957465171814 - rollout_corr/rollout_is_std:0.03981738165020943 - rollout_corr/rollout_is_eff_sample_size:0.9984166697314456 - rollout_corr/rollout_is_seq_mean:0.9997422695159912 - rollout_corr/rollout_is_seq_std:0.0028411736711859703 - rollout_corr/rollout_is_seq_max:1.0074349641799927 - rollout_corr/rollout_is_seq_min:0.9918615221977234 - rollout_corr/rollout_is_seq_max_deviation:0.008138477802276611 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3577020564116538) - rollout_corr/training_log_ppl:np.float64(0.3013367626699619) - rollout_corr/kl:np.float64(0.0011281410026526828) - rollout_corr/k3_kl:np.float64(0.000916888057986398) - rollout_corr/rollout_ppl:np.float64(1.3561205281876028) - rollout_corr/rollout_log_ppl:np.float64(0.3002086222113576) - rollout_corr/log_ppl_diff:np.float64(0.0011281404586043209) - rollout_corr/log_ppl_abs_diff:np.float64(0.00256365115637891) - rollout_corr/log_ppl_diff_max:np.float64(0.008949041366577148) - rollout_corr/log_ppl_diff_min:np.float64(-0.009114667773246765) - rollout_corr/ppl_ratio:np.float64(1.0011336989700794) - rollout_corr/chi2_token:np.float64(0.0014092018827795982) - rollout_corr/chi2_seq:np.float64(0.8297181797679514) - actor/pg_loss:np.float64(0.005181095330044627) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006262393026190693) - actor/ppo_kl:np.float64(4.9855951789634645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(97563.27732459153) - self_distillation/token_reweight_w_std:np.float64(1470502.0369112268) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995444938540459) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10749436303740367) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.4946809858083725) - perf/mfu/actor:np.float64(0.03312182529882031) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.53104400634766) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002026108792051673 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002026108792051673 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/m
(TaskRunner pid=2026458) Training Progress: 46%|████▌ | 46/100 [33:45<30:43, 34.13s/it]
(TaskRunner pid=2026458) step:47 - global_seqlen/min:14094 - global_seqlen/max:21506 - global_seqlen/minmax_diff:7412 - global_seqlen/balanced_min:18641 - global_seqlen/balanced_max:18644 - global_seqlen/mean:18642.625 - actor/entropy:0.27096572518348694 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7320154905319214 - training/rollout_probs_diff_mean:0.005300942808389664 - training/rollout_probs_diff_std:0.015477053821086884 - training/rollout_actor_probs_pearson_corr:0.9979209303855896 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0002624988555908 - rollout_corr/rollout_is_ratio_fraction_high:1.350238289887784e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00017553097859490663 - rollout_corr/rollout_is_max:5.250345230102539 - rollout_corr/rollout_is_min:0.1398765593767166 - rollout_corr/rollout_is_std:0.041101276874542236 - rollout_corr/rollout_is_eff_sample_size:0.9983143657831011 - rollout_corr/rollout_is_seq_mean:1.0002628564834595 - rollout_corr/rollout_is_seq_std:0.0028906052466481924 - rollout_corr/rollout_is_seq_max:1.0110986232757568 - rollout_corr/rollout_is_seq_min:0.9913266897201538 - rollout_corr/rollout_is_seq_max_deviation:0.011098623275756836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.33038782607764) - rollout_corr/training_log_ppl:np.float64(0.2808810631977394) - rollout_corr/kl:np.float64(0.0008357813613741882) - rollout_corr/k3_kl:np.float64(0.0010391420327664491) - rollout_corr/rollout_ppl:np.float64(1.329236305784434) - rollout_corr/rollout_log_ppl:np.float64(0.28004528186284006) - rollout_corr/log_ppl_diff:np.float64(0.0008357813348993659) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023171580396592617) - rollout_corr/log_ppl_diff_max:np.float64(0.011047214269638062) - rollout_corr/log_ppl_diff_min:np.float64(-0.006683439016342163) - rollout_corr/ppl_ratio:np.float64(1.0008402362000197) - rollout_corr/chi2_token:np.float64(0.002344296546652913) - rollout_corr/chi2_seq:np.float64(0.7856939071789384) - actor/pg_loss:np.float64(0.0035682206507772207) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00080771961438586) - actor/ppo_kl:np.float64(0.000137977912149978) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(180196.61803952185) - self_distillation/token_reweight_w_std:np.float64(1850028.0704001258) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998111962340772) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09769563036388718) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.4677669107913971) - perf/mfu/actor:np.float64(0.03410809941893227) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.31404495239258) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003762101521715522 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003762101521715522 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response
(TaskRunner pid=2026458) Training Progress: 47%|████▋ | 47/100 [34:17<29:42, 33.63s/it]
(TaskRunner pid=2026458) step:48 - global_seqlen/min:14172 - global_seqlen/max:18922 - global_seqlen/minmax_diff:4750 - global_seqlen/balanced_min:17273 - global_seqlen/balanced_max:17276 - global_seqlen/mean:17275.125 - actor/entropy:0.28335925936698914 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2923179566860199 - training/rollout_probs_diff_mean:0.005517815239727497 - training/rollout_probs_diff_std:0.0134055744856596 - training/rollout_actor_probs_pearson_corr:0.9984236359596252 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999392032623291 - rollout_corr/rollout_is_ratio_fraction_high:1.4249682863010094e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.8499365726020187e-05 - rollout_corr/rollout_is_max:2.0768961906433105 - rollout_corr/rollout_is_min:0.30299994349479675 - rollout_corr/rollout_is_std:0.04144066944718361 - rollout_corr/rollout_is_eff_sample_size:0.9982854963367315 - rollout_corr/rollout_is_seq_mean:0.9998867511749268 - rollout_corr/rollout_is_seq_std:0.002759289462119341 - rollout_corr/rollout_is_seq_max:1.0119271278381348 - rollout_corr/rollout_is_seq_min:0.9927634000778198 - rollout_corr/rollout_is_seq_max_deviation:0.011927127838134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3371913330629468) - rollout_corr/training_log_ppl:np.float64(0.28653379750903696) - rollout_corr/kl:np.float64(0.0009971100477166317) - rollout_corr/k3_kl:np.float64(0.0009379864269476457) - rollout_corr/rollout_ppl:np.float64(1.3358645238913596) - rollout_corr/rollout_log_ppl:np.float64(0.28553668738459237) - rollout_corr/log_ppl_diff:np.float64(0.000997110124444589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023510434839408845) - rollout_corr/log_ppl_diff_max:np.float64(0.009961530566215515) - rollout_corr/log_ppl_diff_min:np.float64(-0.00725024938583374) - rollout_corr/ppl_ratio:np.float64(1.0010016579180956) - rollout_corr/chi2_token:np.float64(0.0017369601409882307) - rollout_corr/chi2_seq:np.float64(0.5906375367194414) - actor/pg_loss:np.float64(0.0036190334940329194) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005808295049973822) - actor/ppo_kl:np.float64(-3.3437734998642554e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(104527.51239224058) - self_distillation/token_reweight_w_std:np.float64(1541202.450584725) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000157124362886) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09064709232188761) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.42223525047302246) - perf/mfu/actor:np.float64(0.031950463769025786) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.0163230895996) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019717998802661896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019717998802661896 - critic/returns/max:0.875 - critic/returns/min:-0.8
(TaskRunner pid=2026458) Training Progress: 48%|████▊ | 48/100 [34:46<28:01, 32.34s/it]
(TaskRunner pid=2026458) step:49 - global_seqlen/min:17224 - global_seqlen/max:19716 - global_seqlen/minmax_diff:2492 - global_seqlen/balanced_min:18546 - global_seqlen/balanced_max:18550 - global_seqlen/mean:18548.75 - actor/entropy:0.27977773547172546 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3583575189113617 - training/rollout_probs_diff_mean:0.005447280127555132 - training/rollout_probs_diff_std:0.013272756710648537 - training/rollout_actor_probs_pearson_corr:0.9984596967697144 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9998903274536133 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.59693042607978e-05 - rollout_corr/rollout_is_max:1.740144968032837 - rollout_corr/rollout_is_min:0.2820626199245453 - rollout_corr/rollout_is_std:0.04080290347337723 - rollout_corr/rollout_is_eff_sample_size:0.9983375338303951 - rollout_corr/rollout_is_seq_mean:0.9998434782028198 - rollout_corr/rollout_is_seq_std:0.0024030529893934727 - rollout_corr/rollout_is_seq_max:1.0073314905166626 - rollout_corr/rollout_is_seq_min:0.9888448715209961 - rollout_corr/rollout_is_seq_max_deviation:0.011155128479003906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3396195233799517) - rollout_corr/training_log_ppl:np.float64(0.28772373546962626) - rollout_corr/kl:np.float64(0.0007274425354566461) - rollout_corr/k3_kl:np.float64(0.0009657581270516857) - rollout_corr/rollout_ppl:np.float64(1.338642387650907) - rollout_corr/rollout_log_ppl:np.float64(0.2869962909608148) - rollout_corr/log_ppl_diff:np.float64(0.0007274445088114589) - rollout_corr/log_ppl_abs_diff:np.float64(0.002191261184634641) - rollout_corr/log_ppl_diff_max:np.float64(0.011016637086868286) - rollout_corr/log_ppl_diff_min:np.float64(-0.009674102067947388) - rollout_corr/ppl_ratio:np.float64(1.0007315212860703) - rollout_corr/chi2_token:np.float64(0.002396710915490985) - rollout_corr/chi2_seq:np.float64(1.0418477742932737) - actor/pg_loss:np.float64(0.003264832077547908) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005003044952900382) - actor/ppo_kl:np.float64(-0.0002956932246789279) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(133482.32340499433) - self_distillation/token_reweight_w_std:np.float64(1837865.312099801) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993252530694008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07909147051395848) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.44828861951828003) - perf/mfu/actor:np.float64(0.03315053423820138) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(151.27535438537598) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005617484916001558 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005617484916001558 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length
(TaskRunner pid=2026458) Training Progress: 49%|████▉ | 49/100 [35:17<27:09, 31.95s/it]
(TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50}
(TaskRunner pid=2026458) validation generation end
(TaskRunner pid=2026458) [prompt] system
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2026458) <reasoning>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) user
(TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2026458) Please reason step by step.
(TaskRunner pid=2026458) assistant
(TaskRunner pid=2026458) <think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) </think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) [response] <reasoning>
(TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to identify which of the given options matches the structure of the reactant that would form this product through a chemical reaction.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) The product contains a structure with a substituted benzene ring (Cc2ccc(Cl)cc2) and a trifluoromethyl group (C(F)(F)F). The correct reactant should include these key functional groups and match the overall structure of the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option A contains a structure with a substituted benzene ring (c2ccccc2) and a nitro group (N), which is consistent with the product's structure. It also includes a chlorine atom and a trifluoromethyl group, making it a strong candidate for the correct reactant.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option B includes a chlorine atom and a substituted benzene ring, but it lacks the trifluoromethyl group and does not match the structure of the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option C includes a substituted benzene ring and a trifluoromethyl group, but it has a different functional group (an oxygen atom) that is not present in the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option D includes a substituted benzene ring and a trifluoromethyl group, but it has a different functional group (a double bond) that is not present in the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it contains the necessary functional groups and matches the structure of the product.
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458) [ground_truth] A
(TaskRunner pid=2026458) [score] 1.0
(TaskRunner pid=2026458) [acc] 1.0
(TaskRunner pid=2026458) [pred] A
(TaskRunner pid=2026458) [incorrect_format] 1
(TaskRunner pid=2026458) [feedback]
(TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50
(WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor
(TaskRunner pid=2026458) step:50 - global_seqlen/min:15714 - global_seqlen/max:24076 - global_seqlen/minmax_diff:8362 - global_seqlen/balanced_min:18055 - global_seqlen/balanced_max:18058 - global_seqlen/mean:18056.5 - actor/entropy:0.24377241730690002 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36303967237472534 - training/rollout_probs_diff_mean:0.005148114170879126 - training/rollout_probs_diff_std:0.013693224638700485 - training/rollout_actor_probs_pearson_corr:0.9982551336288452 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:2.7458949261927046e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.864737224532291e-05 - rollout_corr/rollout_is_max:2.6678342819213867 - rollout_corr/rollout_is_min:0.17930372059345245 - rollout_corr/rollout_is_std:0.04035785421729088 - rollout_corr/rollout_is_eff_sample_size:0.9983741296547397 - rollout_corr/rollout_is_seq_mean:1.0000826120376587 - rollout_corr/rollout_is_seq_std:0.002955564297735691 - rollout_corr/rollout_is_seq_max:1.0125819444656372 - rollout_corr/rollout_is_seq_min:0.9909312725067139 - rollout_corr/rollout_is_seq_max_deviation:0.012581944465637207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3021241263486445) - rollout_corr/training_log_ppl:np.float64(0.26025468575244304) - rollout_corr/kl:np.float64(0.0007726539755310569) - rollout_corr/k3_kl:np.float64(0.0010959870580791176) - rollout_corr/rollout_ppl:np.float64(1.3010970023460686) - rollout_corr/rollout_log_ppl:np.float64(0.2594820303056622) - rollout_corr/log_ppl_diff:np.float64(0.0007726554467808455) - rollout_corr/log_ppl_abs_diff:np.float64(0.00248320066020824) - rollout_corr/log_ppl_diff_max:np.float64(0.010783776640892029) - rollout_corr/log_ppl_diff_min:np.float64(-0.014668971300125122) - rollout_corr/ppl_ratio:np.float64(1.0007780094165355) - rollout_corr/chi2_token:np.float64(0.002904584165662527) - rollout_corr/chi2_seq:np.float64(1.7037486750632524) - actor/pg_loss:np.float64(0.00390949088614434) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012861561140198319) - actor/ppo_kl:np.float64(-2.874969411337247e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(303182.6307455059) - self_distillation/token_reweight_w_std:np.float64(3853282.430746559) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001452625496313) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08799264291883446) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.9146713092923164) - perf/mfu/actor:np.float64(0.03308314844340515) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.47300720214844) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/reward/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1360982101557332)
(TaskRunner pid=2026458) Training Progress: 50%|█████ | 50/100 [36:59<44:06, 52.94s/it]
(TaskRunner pid=2026458) step:51 - global_seqlen/min:17642 - global_seqlen/max:21320 - global_seqlen/minmax_diff:3678 - global_seqlen/balanced_min:19266 - global_seqlen/balanced_max:19274 - global_seqlen/mean:19272.125 - actor/entropy:0.27418410778045654 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3186328411102295 - training/rollout_probs_diff_mean:0.005429690238088369 - training/rollout_probs_diff_std:0.013169948011636734 - training/rollout_actor_probs_pearson_corr:0.9984568953514099 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999269247055054 - rollout_corr/rollout_is_ratio_fraction_high:2.6965442884829827e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6965442884829827e-05 - rollout_corr/rollout_is_max:2.9393632411956787 - rollout_corr/rollout_is_min:0.18107180297374725 - rollout_corr/rollout_is_std:0.04137301445007324 - rollout_corr/rollout_is_eff_sample_size:0.9982909612066878 - rollout_corr/rollout_is_seq_mean:0.9999714493751526 - rollout_corr/rollout_is_seq_std:0.0026941560208797455 - rollout_corr/rollout_is_seq_max:1.0119987726211548 - rollout_corr/rollout_is_seq_min:0.9906357526779175 - rollout_corr/rollout_is_seq_max_deviation:0.011998772621154785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3326939661055803) - rollout_corr/training_log_ppl:np.float64(0.283328754856484) - rollout_corr/kl:np.float64(0.001354399174752885) - rollout_corr/k3_kl:np.float64(0.0011438059663078093) - rollout_corr/rollout_ppl:np.float64(1.3308947938494384) - rollout_corr/rollout_log_ppl:np.float64(0.2819743549771374) - rollout_corr/log_ppl_diff:np.float64(0.0013543998793466017) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025673647614894435) - rollout_corr/log_ppl_diff_max:np.float64(0.017022624611854553) - rollout_corr/log_ppl_diff_min:np.float64(-0.007628902792930603) - rollout_corr/ppl_ratio:np.float64(1.001360428519547) - rollout_corr/chi2_token:np.float64(0.0018329326994717121) - rollout_corr/chi2_seq:np.float64(1.2063880120404065) - actor/pg_loss:np.float64(0.006517422734759748) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015835570143281075) - actor/ppo_kl:np.float64(0.0004230728660630234) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(238186.0888677421) - self_distillation/token_reweight_w_std:np.float64(3422829.743828913) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022471062839031) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1062716192973312) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.5796995311975479) - perf/mfu/actor:np.float64(0.035632098241036) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.68563079833984) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010413717478513718 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010413717478513718 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_l
(TaskRunner pid=2026458) Training Progress: 51%|█████ | 51/100 [37:29<37:36, 46.06s/it]
(TaskRunner pid=2026458) step:52 - global_seqlen/min:14188 - global_seqlen/max:21038 - global_seqlen/minmax_diff:6850 - global_seqlen/balanced_min:18334 - global_seqlen/balanced_max:18335 - global_seqlen/mean:18334.5 - actor/entropy:0.2573181688785553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730220556259155 - training/rollout_probs_diff_mean:0.005135765764862299 - training/rollout_probs_diff_std:0.012896839529275894 - training/rollout_actor_probs_pearson_corr:0.9984732270240784 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9999938011169434 - rollout_corr/rollout_is_ratio_fraction_high:2.519780355214607e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.519780355214607e-05 - rollout_corr/rollout_is_max:2.1215898990631104 - rollout_corr/rollout_is_min:0.3974515199661255 - rollout_corr/rollout_is_std:0.040721017867326736 - rollout_corr/rollout_is_eff_sample_size:0.9983446626766718 - rollout_corr/rollout_is_seq_mean:0.9999977946281433 - rollout_corr/rollout_is_seq_std:0.0026824194937944412 - rollout_corr/rollout_is_seq_max:1.0067678689956665 - rollout_corr/rollout_is_seq_min:0.9906572103500366 - rollout_corr/rollout_is_seq_max_deviation:0.009342789649963379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3137918766587973) - rollout_corr/training_log_ppl:np.float64(0.2694710589275928) - rollout_corr/kl:np.float64(0.0010667287329408914) - rollout_corr/k3_kl:np.float64(0.0010078331736167456) - rollout_corr/rollout_ppl:np.float64(1.3123429850675166) - rollout_corr/rollout_log_ppl:np.float64(0.26840433149482124) - rollout_corr/log_ppl_diff:np.float64(0.0010667274327715859) - rollout_corr/log_ppl_abs_diff:np.float64(0.002380369245656766) - rollout_corr/log_ppl_diff_max:np.float64(0.010203629732131958) - rollout_corr/log_ppl_diff_min:np.float64(-0.006238669157028198) - rollout_corr/ppl_ratio:np.float64(1.0010714137461036) - rollout_corr/chi2_token:np.float64(0.001948290504515171) - rollout_corr/chi2_seq:np.float64(0.5412429641000926) - actor/pg_loss:np.float64(0.003885703510604799) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005619293583549734) - actor/ppo_kl:np.float64(0.00019392038652732424) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(186537.93477050052) - self_distillation/token_reweight_w_std:np.float64(2011255.6988170468) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005899087991565) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06515720163588412) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4466311186552048) - perf/mfu/actor:np.float64(0.03402000847749656) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.35942840576172) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01099569071084261 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01099569071084261 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/me
(TaskRunner pid=2026458) Training Progress: 52%|█████▏ | 52/100 [37:59<33:00, 41.27s/it]
(TaskRunner pid=2026458) step:53 - global_seqlen/min:17379 - global_seqlen/max:20313 - global_seqlen/minmax_diff:2934 - global_seqlen/balanced_min:18481 - global_seqlen/balanced_max:18482 - global_seqlen/mean:18481.75 - actor/entropy:0.278646856546402 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462908267974854 - training/rollout_probs_diff_mean:0.005563129670917988 - training/rollout_probs_diff_std:0.013384878635406494 - training/rollout_actor_probs_pearson_corr:0.9984221458435059 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:0.9999546408653259 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.5409334891010076e-05 - rollout_corr/rollout_is_max:1.8313579559326172 - rollout_corr/rollout_is_min:0.0737028419971466 - rollout_corr/rollout_is_std:0.04174019396305084 - rollout_corr/rollout_is_eff_sample_size:0.9982605487774039 - rollout_corr/rollout_is_seq_mean:0.9999428391456604 - rollout_corr/rollout_is_seq_std:0.0028358809649944305 - rollout_corr/rollout_is_seq_max:1.0075714588165283 - rollout_corr/rollout_is_seq_min:0.9910898804664612 - rollout_corr/rollout_is_seq_max_deviation:0.008910119533538818 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.338077249005437) - rollout_corr/training_log_ppl:np.float64(0.2873227519157808) - rollout_corr/kl:np.float64(0.0011784572657376202) - rollout_corr/k3_kl:np.float64(0.0013956419594478575) - rollout_corr/rollout_ppl:np.float64(1.3364087981171906) - rollout_corr/rollout_log_ppl:np.float64(0.2861442923604045) - rollout_corr/log_ppl_diff:np.float64(0.001178459555376321) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027842955314554274) - rollout_corr/log_ppl_diff_max:np.float64(0.0132656991481781) - rollout_corr/log_ppl_diff_min:np.float64(-0.010441526770591736) - rollout_corr/ppl_ratio:np.float64(1.0011851615272462) - rollout_corr/chi2_token:np.float64(0.003180387197062373) - rollout_corr/chi2_seq:np.float64(1.2029992926400155) - actor/pg_loss:np.float64(0.003771678893826902) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009931258632605022) - actor/ppo_kl:np.float64(0.00019185108615538127) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(52868.786943790736) - self_distillation/token_reweight_w_std:np.float64(637934.6799204362) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990235811565071) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07278861297527328) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.44415684044361115) - perf/mfu/actor:np.float64(0.0346355208538403) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.01051712036133) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006060396321117878 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006060396321117878 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:281.9921875 - respo
(TaskRunner pid=2026458) Training Progress: 53%|█████▎ | 53/100 [38:29<29:31, 37.70s/it]
(TaskRunner pid=2026458) step:54 - global_seqlen/min:15111 - global_seqlen/max:18863 - global_seqlen/minmax_diff:3752 - global_seqlen/balanced_min:17356 - global_seqlen/balanced_max:17358 - global_seqlen/mean:17357.625 - actor/entropy:0.2270389199256897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413564920425415 - training/rollout_probs_diff_mean:0.005126654170453548 - training/rollout_probs_diff_std:0.01343945786356926 - training/rollout_actor_probs_pearson_corr:0.9981903433799744 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0001782178878784 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.876001235563308e-05 - rollout_corr/rollout_is_max:1.8594493865966797 - rollout_corr/rollout_is_min:0.35453474521636963 - rollout_corr/rollout_is_std:0.04036819189786911 - rollout_corr/rollout_is_eff_sample_size:0.998373654367098 - rollout_corr/rollout_is_seq_mean:1.000128984451294 - rollout_corr/rollout_is_seq_std:0.002654197160154581 - rollout_corr/rollout_is_seq_max:1.0092252492904663 - rollout_corr/rollout_is_seq_min:0.9911713600158691 - rollout_corr/rollout_is_seq_max_deviation:0.009225249290466309 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2745554209686816) - rollout_corr/training_log_ppl:np.float64(0.23950671087368391) - rollout_corr/kl:np.float64(0.0011859710301269644) - rollout_corr/k3_kl:np.float64(0.0012516252674004136) - rollout_corr/rollout_ppl:np.float64(1.2729601715691388) - rollout_corr/rollout_log_ppl:np.float64(0.23832074020174332) - rollout_corr/log_ppl_diff:np.float64(0.001185970671940595) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027243185904808342) - rollout_corr/log_ppl_diff_max:np.float64(0.01128050684928894) - rollout_corr/log_ppl_diff_min:np.float64(-0.011040359735488892) - rollout_corr/ppl_ratio:np.float64(1.0011925832368433) - rollout_corr/chi2_token:np.float64(0.0025079476181417704) - rollout_corr/chi2_seq:np.float64(1.487264727940783) - actor/pg_loss:np.float64(0.0043368348851799965) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017240041906916304) - actor/ppo_kl:np.float64(0.0004469881809043841) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(275413.10884484975) - self_distillation/token_reweight_w_std:np.float64(3763164.204352789) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020550324115902) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08829297052579932) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.5274116396903992) - perf/mfu/actor:np.float64(0.03254539451835227) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.84357070922852) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0047957319766283035 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0047957319766283035 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:271.6445
(TaskRunner pid=2026458) Training Progress: 54%|█████▍ | 54/100 [38:58<26:55, 35.13s/it]
(TaskRunner pid=2026458) step:55 - global_seqlen/min:15554 - global_seqlen/max:21932 - global_seqlen/minmax_diff:6378 - global_seqlen/balanced_min:18814 - global_seqlen/balanced_max:18817 - global_seqlen/mean:18816.625 - actor/entropy:0.22956356406211853 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9232618808746338 - training/rollout_probs_diff_mean:0.005393934901803732 - training/rollout_probs_diff_std:0.014924020506441593 - training/rollout_actor_probs_pearson_corr:0.9978342056274414 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000108242034912 - rollout_corr/rollout_is_ratio_fraction_high:4.133313268539496e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.133313268539496e-05 - rollout_corr/rollout_is_max:2.1670196056365967 - rollout_corr/rollout_is_min:0.04885798692703247 - rollout_corr/rollout_is_std:0.042956262826919556 - rollout_corr/rollout_is_eff_sample_size:0.9981586331310626 - rollout_corr/rollout_is_seq_mean:1.0001986026763916 - rollout_corr/rollout_is_seq_std:0.002816676627844572 - rollout_corr/rollout_is_seq_max:1.0112348794937134 - rollout_corr/rollout_is_seq_min:0.9917329549789429 - rollout_corr/rollout_is_seq_max_deviation:0.011234879493713379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2765053128823638) - rollout_corr/training_log_ppl:np.float64(0.24006979388650507) - rollout_corr/kl:np.float64(0.0010645344639215182) - rollout_corr/k3_kl:np.float64(0.0011318304883047858) - rollout_corr/rollout_ppl:np.float64(1.275088603142649) - rollout_corr/rollout_log_ppl:np.float64(0.23900525833596475) - rollout_corr/log_ppl_diff:np.float64(0.0010645355505403131) - rollout_corr/log_ppl_abs_diff:np.float64(0.002453350374707952) - rollout_corr/log_ppl_diff_max:np.float64(0.011057406663894653) - rollout_corr/log_ppl_diff_min:np.float64(-0.007540911436080933) - rollout_corr/ppl_ratio:np.float64(1.0010695066303015) - rollout_corr/chi2_token:np.float64(0.0023642820306122303) - rollout_corr/chi2_seq:np.float64(0.9924367545172572) - actor/pg_loss:np.float64(0.005413581035099924) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001406622541253455) - actor/ppo_kl:np.float64(0.0002698558086962066) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(295807.0609528662) - self_distillation/token_reweight_w_std:np.float64(3781227.560846627) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014416894409806) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09695331199327484) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5869030505418777) - perf/mfu/actor:np.float64(0.03432748799407685) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.77687454223633) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014518261887133121 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014518261887133121 - critic/returns/max:0.875 - critic/returns/min:-0.875 - resp
(TaskRunner pid=2026458) Training Progress: 55%|█████▌ | 55/100 [39:28<25:11, 33.60s/it]
(TaskRunner pid=2026458) step:56 - global_seqlen/min:15183 - global_seqlen/max:19968 - global_seqlen/minmax_diff:4785 - global_seqlen/balanced_min:16851 - global_seqlen/balanced_max:16856 - global_seqlen/mean:16853.75 - actor/entropy:0.23840856552124023 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032567977905273 - training/rollout_probs_diff_mean:0.0051898774690926075 - training/rollout_probs_diff_std:0.013416245579719543 - training/rollout_actor_probs_pearson_corr:0.9982264637947083 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000948905944824 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.9873039238736965e-05 - rollout_corr/rollout_is_max:1.868481159210205 - rollout_corr/rollout_is_min:0.42863455414772034 - rollout_corr/rollout_is_std:0.03991232439875603 - rollout_corr/rollout_is_eff_sample_size:0.9984097775176272 - rollout_corr/rollout_is_seq_mean:1.0001156330108643 - rollout_corr/rollout_is_seq_std:0.002804977586492896 - rollout_corr/rollout_is_seq_max:1.008453607559204 - rollout_corr/rollout_is_seq_min:0.9917566776275635 - rollout_corr/rollout_is_seq_max_deviation:0.008453607559204102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.288645532913506) - rollout_corr/training_log_ppl:np.float64(0.25009057341958396) - rollout_corr/kl:np.float64(0.001021610375019577) - rollout_corr/k3_kl:np.float64(0.0012247983307815957) - rollout_corr/rollout_ppl:np.float64(1.2872521225363016) - rollout_corr/rollout_log_ppl:np.float64(0.2490689625556115) - rollout_corr/log_ppl_diff:np.float64(0.0010216108639724553) - rollout_corr/log_ppl_abs_diff:np.float64(0.002677871671039611) - rollout_corr/log_ppl_diff_max:np.float64(0.011799484491348267) - rollout_corr/log_ppl_diff_min:np.float64(-0.013348877429962158) - rollout_corr/ppl_ratio:np.float64(1.001028069993481) - rollout_corr/chi2_token:np.float64(0.0027951118536293507) - rollout_corr/chi2_seq:np.float64(0.911775927990675) - actor/pg_loss:np.float64(0.0038659967249259353) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001200501193125092) - actor/ppo_kl:np.float64(0.0002928259359418206) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(173579.36585247447) - self_distillation/token_reweight_w_std:np.float64(2190372.348648581) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999903631862253) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08299089543288574) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.5205514430999756) - perf/mfu/actor:np.float64(0.03136809590233023) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.65859985351562) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00015309933223761618 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00015309933223761618 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length
(TaskRunner pid=2026458) Training Progress: 56%|█████▌ | 56/100 [39:58<23:46, 32.42s/it]
(TaskRunner pid=2026458) step:57 - global_seqlen/min:15145 - global_seqlen/max:21004 - global_seqlen/minmax_diff:5859 - global_seqlen/balanced_min:17327 - global_seqlen/balanced_max:17330 - global_seqlen/mean:17329.125 - actor/entropy:0.2190825492143631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744122624397278 - training/rollout_probs_diff_mean:0.005394987761974335 - training/rollout_probs_diff_std:0.014672977849841118 - training/rollout_actor_probs_pearson_corr:0.9977869987487793 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0002177953720093 - rollout_corr/rollout_is_ratio_fraction_high:9.200055501423776e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013800083252135664 - rollout_corr/rollout_is_max:3.8171095848083496 - rollout_corr/rollout_is_min:0.15978100895881653 - rollout_corr/rollout_is_std:0.04392360523343086 - rollout_corr/rollout_is_eff_sample_size:0.9980752630793054 - rollout_corr/rollout_is_seq_mean:1.0002235174179077 - rollout_corr/rollout_is_seq_std:0.0027234358713030815 - rollout_corr/rollout_is_seq_max:1.0101075172424316 - rollout_corr/rollout_is_seq_min:0.9909296035766602 - rollout_corr/rollout_is_seq_max_deviation:0.01010751724243164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2523039323277771) - rollout_corr/training_log_ppl:np.float64(0.22247063313261606) - rollout_corr/kl:np.float64(0.0011533432969415003) - rollout_corr/k3_kl:np.float64(0.0011055539546305226) - rollout_corr/rollout_ppl:np.float64(1.2508012815378606) - rollout_corr/rollout_log_ppl:np.float64(0.22131728671956807) - rollout_corr/log_ppl_diff:np.float64(0.0011533464130479842) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025634898629505187) - rollout_corr/log_ppl_diff_max:np.float64(0.016222700476646423) - rollout_corr/log_ppl_diff_min:np.float64(-0.007622018456459045) - rollout_corr/ppl_ratio:np.float64(1.0011591671500355) - rollout_corr/chi2_token:np.float64(0.0020452477037906647) - rollout_corr/chi2_seq:np.float64(0.5157537555787712) - actor/pg_loss:np.float64(0.003759522340260446) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008975071282293356) - actor/ppo_kl:np.float64(0.00035579913428307464) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(236077.81382388272) - self_distillation/token_reweight_w_std:np.float64(2308889.815033672) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994992383290082) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0630278438620735) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.43273352086544037) - perf/mfu/actor:np.float64(0.031510354454649704) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.82762956619263) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027676832396537066 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027676832396537066 - critic/returns/max:0.875 - critic/returns/min:-0.875 - res
(TaskRunner pid=2026458) Training Progress: 57%|█████▋ | 57/100 [40:28<22:46, 31.77s/it]
(TaskRunner pid=2026458) step:58 - global_seqlen/min:14734 - global_seqlen/max:19599 - global_seqlen/minmax_diff:4865 - global_seqlen/balanced_min:17102 - global_seqlen/balanced_max:17103 - global_seqlen/mean:17102.75 - actor/entropy:0.23608146607875824 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5141049027442932 - training/rollout_probs_diff_mean:0.005427653901278973 - training/rollout_probs_diff_std:0.013945525512099266 - training/rollout_actor_probs_pearson_corr:0.9981228709220886 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999080896377563 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5163001990003977e-05 - rollout_corr/rollout_is_max:1.724382996559143 - rollout_corr/rollout_is_min:0.16556662321090698 - rollout_corr/rollout_is_std:0.04180582985281944 - rollout_corr/rollout_is_eff_sample_size:0.9982550842404107 - rollout_corr/rollout_is_seq_mean:0.9999476671218872 - rollout_corr/rollout_is_seq_std:0.002901083789765835 - rollout_corr/rollout_is_seq_max:1.00826895236969 - rollout_corr/rollout_is_seq_min:0.9917892217636108 - rollout_corr/rollout_is_seq_max_deviation:0.008268952369689941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2862667385488749) - rollout_corr/training_log_ppl:np.float64(0.24756408171379007) - rollout_corr/kl:np.float64(0.0008958796237372724) - rollout_corr/k3_kl:np.float64(0.0010313347952433105) - rollout_corr/rollout_ppl:np.float64(1.2851230534724891) - rollout_corr/rollout_log_ppl:np.float64(0.24666820294805802) - rollout_corr/log_ppl_diff:np.float64(0.0008958787657320499) - rollout_corr/log_ppl_abs_diff:np.float64(0.002455878769978881) - rollout_corr/log_ppl_diff_max:np.float64(0.014661163091659546) - rollout_corr/log_ppl_diff_min:np.float64(-0.006324887275695801) - rollout_corr/ppl_ratio:np.float64(1.00090107973665) - rollout_corr/chi2_token:np.float64(0.0023643041495233774) - rollout_corr/chi2_seq:np.float64(0.6059110292699188) - actor/pg_loss:np.float64(0.003710375865921378) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011193532754987245) - actor/ppo_kl:np.float64(-8.923532409355062e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(286496.5454768897) - self_distillation/token_reweight_w_std:np.float64(3628499.957205932) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005135682877153) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07967197702964768) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5009380057454109) - perf/mfu/actor:np.float64(0.03152191694270064) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.63263320922852) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016660349210724235 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016660349210724235 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_lengt
(TaskRunner pid=2026458) Training Progress: 58%|█████▊ | 58/100 [40:57<21:38, 30.92s/it]
(TaskRunner pid=2026458) step:59 - global_seqlen/min:15244 - global_seqlen/max:19485 - global_seqlen/minmax_diff:4241 - global_seqlen/balanced_min:16769 - global_seqlen/balanced_max:16771 - global_seqlen/mean:16770.625 - actor/entropy:0.2315026819705963 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5046374797821045 - training/rollout_probs_diff_mean:0.0052477153949439526 - training/rollout_probs_diff_std:0.013823322020471096 - training/rollout_actor_probs_pearson_corr:0.9981176853179932 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:1.0000110864639282 - rollout_corr/rollout_is_ratio_fraction_high:1.5287481801351532e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.114992720540613e-05 - rollout_corr/rollout_is_max:2.5501413345336914 - rollout_corr/rollout_is_min:0.3053041398525238 - rollout_corr/rollout_is_std:0.04093707725405693 - rollout_corr/rollout_is_eff_sample_size:0.998326959562596 - rollout_corr/rollout_is_seq_mean:1.000074863433838 - rollout_corr/rollout_is_seq_std:0.0029152550268918276 - rollout_corr/rollout_is_seq_max:1.0087932348251343 - rollout_corr/rollout_is_seq_min:0.9920890927314758 - rollout_corr/rollout_is_seq_max_deviation:0.008793234825134277 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2814294979907572) - rollout_corr/training_log_ppl:np.float64(0.24554033184540458) - rollout_corr/kl:np.float64(0.0010128226648049576) - rollout_corr/k3_kl:np.float64(0.0010760613429283694) - rollout_corr/rollout_ppl:np.float64(1.280145457945764) - rollout_corr/rollout_log_ppl:np.float64(0.2445275079808198) - rollout_corr/log_ppl_diff:np.float64(0.0010128238645847887) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026435373874846846) - rollout_corr/log_ppl_diff_max:np.float64(0.01640312373638153) - rollout_corr/log_ppl_diff_min:np.float64(-0.01078614592552185) - rollout_corr/ppl_ratio:np.float64(1.001019265735522) - rollout_corr/chi2_token:np.float64(0.0022960123606026173) - rollout_corr/chi2_seq:np.float64(0.9392846249975264) - actor/pg_loss:np.float64(0.002857705345377326) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044364196128299227) - actor/ppo_kl:np.float64(0.0001824798405234418) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(71359.55967221712) - self_distillation/token_reweight_w_std:np.float64(989988.4983619121) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995234010275453) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04141386490664445) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.3792598769068718) - perf/mfu/actor:np.float64(0.03147454843432276) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.65855026245117) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003848623251542449 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:0.003848623251542449 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/m
(TaskRunner pid=2026458) Training Progress: 59%|█████▉ | 59/100 [41:26<20:43, 30.33s/it]
(TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60}
(TaskRunner pid=2026458) validation generation end
(TaskRunner pid=2026458) [prompt] system
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2026458) <reasoning>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) user
(TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2026458) Please reason step by step.
(TaskRunner pid=2026458) assistant
(TaskRunner pid=2026458) <think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) </think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) [response] <reasoning>
(TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to identify which of the given options contains the correct functional groups and structure that match the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) The product contains a nitrogen-containing group (n1) connected to a cyclohexane ring (Cc2ccc(Cl)cc2), with a Cl group and a trifluoromethyl group (C(F)(F)F). The correct reactant must include these key features.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option A contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1) connected to a cyclohexane ring with a Cl group and a trifluoromethyl group. This matches the structure of the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option B contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it does not include the Cl group or the trifluoromethyl group in the correct position.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option C contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it includes a different functional group (Oc2ccc(Cl)c(C(F)(F)F)c2) that does not match the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option D contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it includes a different functional group (CC=C(Cc1ccccc1)C(F)(F)F) that does not match the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Therefore, the correct answer is Option A.
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458) [ground_truth] A
(TaskRunner pid=2026458) [score] 1.0
(TaskRunner pid=2026458) [acc] 1.0
(TaskRunner pid=2026458) [pred] A
(TaskRunner pid=2026458) [incorrect_format] 1
(TaskRunner pid=2026458) [feedback]
(TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60
(WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor
(TaskRunner pid=2026458) step:60 - global_seqlen/min:14552 - global_seqlen/max:17730 - global_seqlen/minmax_diff:3178 - global_seqlen/balanced_min:16889 - global_seqlen/balanced_max:16891 - global_seqlen/mean:16890.25 - actor/entropy:0.258317232131958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5851317644119263 - training/rollout_probs_diff_mean:0.005422248970717192 - training/rollout_probs_diff_std:0.013785075396299362 - training/rollout_actor_probs_pearson_corr:0.9982817769050598 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0001024007797241 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4916022337274626e-05 - rollout_corr/rollout_is_max:1.976409912109375 - rollout_corr/rollout_is_min:0.3921801745891571 - rollout_corr/rollout_is_std:0.041325442492961884 - rollout_corr/rollout_is_eff_sample_size:0.9982955945128663 - rollout_corr/rollout_is_seq_mean:1.000079870223999 - rollout_corr/rollout_is_seq_std:0.002677295822650194 - rollout_corr/rollout_is_seq_max:1.0086641311645508 - rollout_corr/rollout_is_seq_min:0.9929971694946289 - rollout_corr/rollout_is_seq_max_deviation:0.008664131164550781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3128179563209414) - rollout_corr/training_log_ppl:np.float64(0.2678291550837457) - rollout_corr/kl:np.float64(0.001152293861238718) - rollout_corr/k3_kl:np.float64(0.0010784027733734547) - rollout_corr/rollout_ppl:np.float64(1.3112971791997552) - rollout_corr/rollout_log_ppl:np.float64(0.2666768589988351) - rollout_corr/log_ppl_diff:np.float64(0.0011522960849106312) - rollout_corr/log_ppl_abs_diff:np.float64(0.002510662190616131) - rollout_corr/log_ppl_diff_max:np.float64(0.009669288992881775) - rollout_corr/log_ppl_diff_min:np.float64(-0.007052525877952576) - rollout_corr/ppl_ratio:np.float64(1.0011574435047805) - rollout_corr/chi2_token:np.float64(0.002051107119768858) - rollout_corr/chi2_seq:np.float64(0.5519429405685514) - actor/pg_loss:np.float64(0.003786680055782199) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008521524032403249) - actor/ppo_kl:np.float64(0.0003417033792416646) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(148988.88128135214) - self_distillation/token_reweight_w_std:np.float64(1866554.9818183868) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014604937750846) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08368598294327967) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.4603109806776047) - perf/mfu/actor:np.float64(0.03142972200463806) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.4005994796753) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/reward/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknow
(TaskRunner pid=2026458) Training Progress: 60%|██████ | 60/100 [43:07<34:24, 51.61s/it]
(TaskRunner pid=2026458) step:61 - global_seqlen/min:14736 - global_seqlen/max:19497 - global_seqlen/minmax_diff:4761 - global_seqlen/balanced_min:17532 - global_seqlen/balanced_max:17536 - global_seqlen/mean:17534.5 - actor/entropy:0.2423284649848938 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49669769406318665 - training/rollout_probs_diff_mean:0.005022233817726374 - training/rollout_probs_diff_std:0.013324090279638767 - training/rollout_actor_probs_pearson_corr:0.9983177781105042 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0002062320709229 - rollout_corr/rollout_is_ratio_fraction_high:5.464779314934276e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0985847590491176e-05 - rollout_corr/rollout_is_max:7.0302934646606445 - rollout_corr/rollout_is_min:0.35777419805526733 - rollout_corr/rollout_is_std:0.03978368639945984 - rollout_corr/rollout_is_eff_sample_size:0.998420353539922 - rollout_corr/rollout_is_seq_mean:1.0001554489135742 - rollout_corr/rollout_is_seq_std:0.002474896376952529 - rollout_corr/rollout_is_seq_max:1.0129971504211426 - rollout_corr/rollout_is_seq_min:0.9919136762619019 - rollout_corr/rollout_is_seq_max_deviation:0.012997150421142578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2963423300534487) - rollout_corr/training_log_ppl:np.float64(0.25556481117382646) - rollout_corr/kl:np.float64(0.0009956922927543133) - rollout_corr/k3_kl:np.float64(0.0010460353189500893) - rollout_corr/rollout_ppl:np.float64(1.2950277719646692) - rollout_corr/rollout_log_ppl:np.float64(0.25456911721266806) - rollout_corr/log_ppl_diff:np.float64(0.0009956939611583948) - rollout_corr/log_ppl_abs_diff:np.float64(0.002202652278356254) - rollout_corr/log_ppl_diff_max:np.float64(0.010044902563095093) - rollout_corr/log_ppl_diff_min:np.float64(-0.009525060653686523) - rollout_corr/ppl_ratio:np.float64(1.0009999251924455) - rollout_corr/chi2_token:np.float64(0.0022548860870301723) - rollout_corr/chi2_seq:np.float64(0.8442793434951454) - actor/pg_loss:np.float64(0.0042004447896033525) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007894714767644473) - actor/ppo_kl:np.float64(0.0002806229531095994) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(132276.08868701942) - self_distillation/token_reweight_w_std:np.float64(1859101.6540320972) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012411118950695) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08409801000379957) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4940867945551872) - perf/mfu/actor:np.float64(0.03230972888329671) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(146.0987777709961) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0024950134102255106 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0024950134102255106 - critic/returns/max:0.875 - critic/returns/min:-0.87
(TaskRunner pid=2026458) Training Progress: 61%|██████ | 61/100 [43:37<29:22, 45.18s/it]
(TaskRunner pid=2026458) step:62 - global_seqlen/min:15907 - global_seqlen/max:20488 - global_seqlen/minmax_diff:4581 - global_seqlen/balanced_min:18196 - global_seqlen/balanced_max:18198 - global_seqlen/mean:18197.0 - actor/entropy:0.26120486855506897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35323476791381836 - training/rollout_probs_diff_mean:0.005538288038223982 - training/rollout_probs_diff_std:0.014037296175956726 - training/rollout_actor_probs_pearson_corr:0.9981968998908997 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.9998036623001099 - rollout_corr/rollout_is_ratio_fraction_high:2.8998956622672267e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014499477401841432 - rollout_corr/rollout_is_max:2.314438819885254 - rollout_corr/rollout_is_min:0.32504311203956604 - rollout_corr/rollout_is_std:0.04294585436582565 - rollout_corr/rollout_is_eff_sample_size:0.9981582768191568 - rollout_corr/rollout_is_seq_mean:0.9998564720153809 - rollout_corr/rollout_is_seq_std:0.0027946694754064083 - rollout_corr/rollout_is_seq_max:1.0084697008132935 - rollout_corr/rollout_is_seq_min:0.9910843372344971 - rollout_corr/rollout_is_seq_max_deviation:0.00891566276550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.316816312726587) - rollout_corr/training_log_ppl:np.float64(0.27135412095231004) - rollout_corr/kl:np.float64(0.0013785573038145316) - rollout_corr/k3_kl:np.float64(0.0011328787384172756) - rollout_corr/rollout_ppl:np.float64(1.3149180496111512) - rollout_corr/rollout_log_ppl:np.float64(0.2699755621724762) - rollout_corr/log_ppl_diff:np.float64(0.0013785587798338383) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027030363271478564) - rollout_corr/log_ppl_diff_max:np.float64(0.010782003402709961) - rollout_corr/log_ppl_diff_min:np.float64(-0.008555859327316284) - rollout_corr/ppl_ratio:np.float64(1.0013843621127307) - rollout_corr/chi2_token:np.float64(0.0016965712420642376) - rollout_corr/chi2_seq:np.float64(0.6767998114228249) - actor/pg_loss:np.float64(0.003519455436617136) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008139853748616588) - actor/ppo_kl:np.float64(0.0002313825499626887) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(140065.4251762007) - self_distillation/token_reweight_w_std:np.float64(1931525.736894751) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999061315320432) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07281633446109481) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.4214628040790558) - perf/mfu/actor:np.float64(0.033835202551722315) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(146.0380401611328) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003894922323524952 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003894922323524952 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_len
(TaskRunner pid=2026458) Training Progress: 62%|██████▏ | 62/100 [44:06<25:32, 40.33s/it]
(TaskRunner pid=2026458) step:63 - global_seqlen/min:15346 - global_seqlen/max:20074 - global_seqlen/minmax_diff:4728 - global_seqlen/balanced_min:17670 - global_seqlen/balanced_max:17672 - global_seqlen/mean:17671.0 - actor/entropy:0.2494240403175354 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44295963644981384 - training/rollout_probs_diff_mean:0.005514265038073063 - training/rollout_probs_diff_std:0.013980090618133545 - training/rollout_actor_probs_pearson_corr:0.9981726408004761 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9999319911003113 - rollout_corr/rollout_is_ratio_fraction_high:1.4685150745208375e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.811090083327144e-05 - rollout_corr/rollout_is_max:2.1109066009521484 - rollout_corr/rollout_is_min:0.21203994750976562 - rollout_corr/rollout_is_std:0.04199644550681114 - rollout_corr/rollout_is_eff_sample_size:0.9982390473577023 - rollout_corr/rollout_is_seq_mean:0.9999494552612305 - rollout_corr/rollout_is_seq_std:0.003092808648943901 - rollout_corr/rollout_is_seq_max:1.0130763053894043 - rollout_corr/rollout_is_seq_min:0.9914883375167847 - rollout_corr/rollout_is_seq_max_deviation:0.013076305389404297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.302847915329039) - rollout_corr/training_log_ppl:np.float64(0.26041782830725424) - rollout_corr/kl:np.float64(0.001262190604840896) - rollout_corr/k3_kl:np.float64(0.0011403532549252304) - rollout_corr/rollout_ppl:np.float64(1.3011768972501159) - rollout_corr/rollout_log_ppl:np.float64(0.2591556366533041) - rollout_corr/log_ppl_diff:np.float64(0.0012621916539501399) - rollout_corr/log_ppl_abs_diff:np.float64(0.002628726389957592) - rollout_corr/log_ppl_diff_max:np.float64(0.010536238551139832) - rollout_corr/log_ppl_diff_min:np.float64(-0.009842857718467712) - rollout_corr/ppl_ratio:np.float64(1.0012680946383625) - rollout_corr/chi2_token:np.float64(0.0019653229974210262) - rollout_corr/chi2_seq:np.float64(0.7706008821260184) - actor/pg_loss:np.float64(0.0034944345243275166) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000709521887529263) - actor/ppo_kl:np.float64(0.00027179806370725146) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(168796.3436012969) - self_distillation/token_reweight_w_std:np.float64(2124951.2078802614) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015958338044584) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0751116150058806) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.4171854108572006) - perf/mfu/actor:np.float64(0.03263190662040404) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.89733123779297) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005943814758211374 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005943814758211374 - critic/returns/max:0.875 - critic/returns/min:-0.875 -
(TaskRunner pid=2026458) Training Progress: 63%|██████▎ | 63/100 [44:36<22:53, 37.11s/it]
(TaskRunner pid=2026458) step:64 - global_seqlen/min:14067 - global_seqlen/max:24197 - global_seqlen/minmax_diff:10130 - global_seqlen/balanced_min:18752 - global_seqlen/balanced_max:25893 - global_seqlen/mean:19646.75 - actor/entropy:0.21532860398292542 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8697976469993591 - training/rollout_probs_diff_mean:0.004864232614636421 - training/rollout_probs_diff_std:0.013850908726453781 - training/rollout_actor_probs_pearson_corr:0.9980413317680359 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0002894401550293 - rollout_corr/rollout_is_ratio_fraction_high:6.0805057728430256e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.4322023818967864e-05 - rollout_corr/rollout_is_max:8.297144889831543 - rollout_corr/rollout_is_min:0.23644955456256866 - rollout_corr/rollout_is_std:0.04067561775445938 - rollout_corr/rollout_is_eff_sample_size:0.9983489400333069 - rollout_corr/rollout_is_seq_mean:1.000409722328186 - rollout_corr/rollout_is_seq_std:0.0031241707038134336 - rollout_corr/rollout_is_seq_max:1.0219839811325073 - rollout_corr/rollout_is_seq_min:0.9937897324562073 - rollout_corr/rollout_is_seq_max_deviation:0.021983981132507324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2946171956136823) - rollout_corr/training_log_ppl:np.float64(0.25466993783356884) - rollout_corr/kl:np.float64(0.0008532866019912433) - rollout_corr/k3_kl:np.float64(0.0010788948769118178) - rollout_corr/rollout_ppl:np.float64(1.2935234676115215) - rollout_corr/rollout_log_ppl:np.float64(0.25381665135137155) - rollout_corr/log_ppl_diff:np.float64(0.0008532864821972908) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022936501272852183) - rollout_corr/log_ppl_diff_max:np.float64(0.00881144404411316) - rollout_corr/log_ppl_diff_min:np.float64(-0.008881300687789917) - rollout_corr/ppl_ratio:np.float64(1.0008577948901802) - rollout_corr/chi2_token:np.float64(0.003156282240524888) - rollout_corr/chi2_seq:np.float64(3.035388365620747) - actor/pg_loss:np.float64(0.0014677918516099453) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004414845861901995) - actor/ppo_kl:np.float64(0.00026685018261352056) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(81162.51946377661) - self_distillation/token_reweight_w_std:np.float64(1170164.887981981) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010531947482377) - self_distillation/token_reweight_w_clip_frac:np.float64(0.049339942750521004) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.263648372143507) - perf/mfu/actor:np.float64(0.03462126720848994) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.74031829833984) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.08103185892105103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.08103185892105103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - respo
(TaskRunner pid=2026458) Training Progress: 64%|██████▍ | 64/100 [45:51<29:10, 48.62s/it]
(TaskRunner pid=2026458) step:65 - global_seqlen/min:15781 - global_seqlen/max:21398 - global_seqlen/minmax_diff:5617 - global_seqlen/balanced_min:18710 - global_seqlen/balanced_max:18715 - global_seqlen/mean:18712.625 - actor/entropy:0.2451825588941574 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43764662742614746 - training/rollout_probs_diff_mean:0.005050553474575281 - training/rollout_probs_diff_std:0.013090502470731735 - training/rollout_actor_probs_pearson_corr:0.9983762502670288 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999321103096008 - rollout_corr/rollout_is_ratio_fraction_high:1.2970000170753337e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.8910002331249416e-05 - rollout_corr/rollout_is_max:2.924093246459961 - rollout_corr/rollout_is_min:0.4824545979499817 - rollout_corr/rollout_is_std:0.040282461792230606 - rollout_corr/rollout_is_eff_sample_size:0.9983795954951445 - rollout_corr/rollout_is_seq_mean:0.9999394416809082 - rollout_corr/rollout_is_seq_std:0.002558288164436817 - rollout_corr/rollout_is_seq_max:1.0071020126342773 - rollout_corr/rollout_is_seq_min:0.9918232560157776 - rollout_corr/rollout_is_seq_max_deviation:0.008176743984222412 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2973304083570838) - rollout_corr/training_log_ppl:np.float64(0.2564336106006522) - rollout_corr/kl:np.float64(0.0012040652124341022) - rollout_corr/k3_kl:np.float64(0.000990912636439134) - rollout_corr/rollout_ppl:np.float64(1.2957068742252886) - rollout_corr/rollout_log_ppl:np.float64(0.2552295460482128) - rollout_corr/log_ppl_diff:np.float64(0.0012040645524393767) - rollout_corr/log_ppl_abs_diff:np.float64(0.002526016061892733) - rollout_corr/log_ppl_diff_max:np.float64(0.011661797761917114) - rollout_corr/log_ppl_diff_min:np.float64(-0.006277412176132202) - rollout_corr/ppl_ratio:np.float64(1.001209291163832) - rollout_corr/chi2_token:np.float64(0.0015423318836838007) - rollout_corr/chi2_seq:np.float64(0.9701050808653235) - actor/pg_loss:np.float64(0.0032792859710752964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007688776604481973) - actor/ppo_kl:np.float64(0.0002768948086071532) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(86791.39557419321) - self_distillation/token_reweight_w_std:np.float64(1084418.8798323106) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982076121959835) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08052845095517114) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.3758349269628525) - perf/mfu/actor:np.float64(0.03501136292112507) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.70735931396484) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0052852751687169075 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0052852751687169075 - critic/returns/max:0.875 - critic/returns/min:-0.875 -
(TaskRunner pid=2026458) Training Progress: 65%|██████▌ | 65/100 [46:22<25:09, 43.12s/it]
(TaskRunner pid=2026458) step:66 - global_seqlen/min:16986 - global_seqlen/max:21356 - global_seqlen/minmax_diff:4370 - global_seqlen/balanced_min:19034 - global_seqlen/balanced_max:19038 - global_seqlen/mean:19037.375 - actor/entropy:0.2407594919204712 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26579251885414124 - training/rollout_probs_diff_mean:0.005038884002715349 - training/rollout_probs_diff_std:0.013142738491296768 - training/rollout_actor_probs_pearson_corr:0.9983550906181335 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:1.000045657157898 - rollout_corr/rollout_is_ratio_fraction_high:2.5642670152592473e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.974934462457895e-05 - rollout_corr/rollout_is_max:2.355177402496338 - rollout_corr/rollout_is_min:0.19801293313503265 - rollout_corr/rollout_is_std:0.04063016548752785 - rollout_corr/rollout_is_eff_sample_size:0.9983520292581156 - rollout_corr/rollout_is_seq_mean:1.0001461505889893 - rollout_corr/rollout_is_seq_std:0.0026480352971702814 - rollout_corr/rollout_is_seq_max:1.0091423988342285 - rollout_corr/rollout_is_seq_min:0.9913557171821594 - rollout_corr/rollout_is_seq_max_deviation:0.009142398834228516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3078779838979244) - rollout_corr/training_log_ppl:np.float64(0.26354357780655846) - rollout_corr/kl:np.float64(0.0010749662497779866) - rollout_corr/k3_kl:np.float64(0.0014432408553943787) - rollout_corr/rollout_ppl:np.float64(1.3064235104247928) - rollout_corr/rollout_log_ppl:np.float64(0.2624686111230403) - rollout_corr/log_ppl_diff:np.float64(0.0010749666835181415) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029108942835591733) - rollout_corr/log_ppl_diff_max:np.float64(0.013920098543167114) - rollout_corr/log_ppl_diff_min:np.float64(-0.009019121527671814) - rollout_corr/ppl_ratio:np.float64(1.0010822340846062) - rollout_corr/chi2_token:np.float64(0.003545147832483053) - rollout_corr/chi2_seq:np.float64(1.548627549316734) - actor/pg_loss:np.float64(0.003003176534548402) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009145650715254305) - actor/ppo_kl:np.float64(0.0003188287759954278) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.546875) - self_distillation/token_reweight_w_mean:np.float64(126959.81397781824) - self_distillation/token_reweight_w_std:np.float64(1629415.2238851967) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008254635613412) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06369021107093431) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.4391012266278267) - perf/mfu/actor:np.float64(0.03551272812953314) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.41056060791016) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004540354944765568 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004540354944765568 - critic/returns/max:0.875 - critic/returns/min:-0.875 - re
(TaskRunner pid=2026458) Training Progress: 66%|██████▌ | 66/100 [46:52<22:19, 39.40s/it]
(TaskRunner pid=2026458) step:67 - global_seqlen/min:15159 - global_seqlen/max:19915 - global_seqlen/minmax_diff:4756 - global_seqlen/balanced_min:18046 - global_seqlen/balanced_max:18050 - global_seqlen/mean:18048.5 - actor/entropy:0.22801370918750763 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9852267503738403 - training/rollout_probs_diff_mean:0.005018836818635464 - training/rollout_probs_diff_std:0.014277998358011246 - training/rollout_actor_probs_pearson_corr:0.9979830384254456 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0002268552780151 - rollout_corr/rollout_is_ratio_fraction_high:2.8704287615255453e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001004650112008676 - rollout_corr/rollout_is_max:2.3428878784179688 - rollout_corr/rollout_is_min:0.0007197936065495014 - rollout_corr/rollout_is_std:0.040458161383867264 - rollout_corr/rollout_is_eff_sample_size:0.9983666439269478 - rollout_corr/rollout_is_seq_mean:1.0003077983856201 - rollout_corr/rollout_is_seq_std:0.0026670496445149183 - rollout_corr/rollout_is_seq_max:1.0164320468902588 - rollout_corr/rollout_is_seq_min:0.9921717047691345 - rollout_corr/rollout_is_seq_max_deviation:0.01643204689025879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2767656035721302) - rollout_corr/training_log_ppl:np.float64(0.24004710461304057) - rollout_corr/kl:np.float64(0.0011609104617287258) - rollout_corr/k3_kl:np.float64(0.001655576513314827) - rollout_corr/rollout_ppl:np.float64(1.2751760962419212) - rollout_corr/rollout_log_ppl:np.float64(0.23888619328499772) - rollout_corr/log_ppl_diff:np.float64(0.0011609113280428573) - rollout_corr/log_ppl_abs_diff:np.float64(0.003236807693610899) - rollout_corr/log_ppl_diff_max:np.float64(0.017100274562835693) - rollout_corr/log_ppl_diff_min:np.float64(-0.02350728213787079) - rollout_corr/ppl_ratio:np.float64(1.0011714671272784) - rollout_corr/chi2_token:np.float64(0.004160782555118203) - rollout_corr/chi2_seq:np.float64(5.24587120115757) - actor/pg_loss:np.float64(0.0036287190159782767) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001126493662923167) - actor/ppo_kl:np.float64(0.0005246336532138685) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(249329.88290784857) - self_distillation/token_reweight_w_std:np.float64(2866482.3437989554) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994784640148282) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06014655635226518) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.471420306712389) - perf/mfu/actor:np.float64(0.033855896624554245) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.65926361083984) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004662652965635061 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004662652965635061 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_len
(TaskRunner pid=2026458) Training Progress: 67%|██████▋ | 67/100 [47:21<19:57, 36.28s/it]
(TaskRunner pid=2026458) step:68 - global_seqlen/min:14735 - global_seqlen/max:19627 - global_seqlen/minmax_diff:4892 - global_seqlen/balanced_min:17371 - global_seqlen/balanced_max:17375 - global_seqlen/mean:17373.75 - actor/entropy:0.26004546880722046 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23114943504333496 - training/rollout_probs_diff_mean:0.005509829614311457 - training/rollout_probs_diff_std:0.013461560010910034 - training/rollout_actor_probs_pearson_corr:0.9983240365982056 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.58984375 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.58984375 - rollout_corr/rollout_is_mean:1.0001424551010132 - rollout_corr/rollout_is_ratio_fraction_high:3.430884680710733e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.14632738486398e-05 - rollout_corr/rollout_is_max:2.094249725341797 - rollout_corr/rollout_is_min:0.4616723358631134 - rollout_corr/rollout_is_std:0.04154840111732483 - rollout_corr/rollout_is_eff_sample_size:0.998277299143981 - rollout_corr/rollout_is_seq_mean:1.0000828504562378 - rollout_corr/rollout_is_seq_std:0.0031394551042467356 - rollout_corr/rollout_is_seq_max:1.0125868320465088 - rollout_corr/rollout_is_seq_min:0.9910873770713806 - rollout_corr/rollout_is_seq_max_deviation:0.012586832046508789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3149044853635132) - rollout_corr/training_log_ppl:np.float64(0.26928867847891524) - rollout_corr/kl:np.float64(0.0007745910923624422) - rollout_corr/k3_kl:np.float64(0.0011792316492460486) - rollout_corr/rollout_ppl:np.float64(1.3139065788127482) - rollout_corr/rollout_log_ppl:np.float64(0.26851408526999876) - rollout_corr/log_ppl_diff:np.float64(0.0007745932089164853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025513150612823665) - rollout_corr/log_ppl_diff_max:np.float64(0.009900659322738647) - rollout_corr/log_ppl_diff_min:np.float64(-0.015614360570907593) - rollout_corr/ppl_ratio:np.float64(1.0007805349305272) - rollout_corr/chi2_token:np.float64(0.0031827117782086134) - rollout_corr/chi2_seq:np.float64(0.9155827462673187) - actor/pg_loss:np.float64(0.0037549110129475594) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007232226507767336) - actor/ppo_kl:np.float64(-4.5092836657545377e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.58984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(174603.2610843731) - self_distillation/token_reweight_w_std:np.float64(2132750.0757954237) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9986913346219808) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0688194558606483) - self_distillation/empty_target_batch:np.float64(0.41015625) - actor/grad_norm:np.float64(0.4323424845933914) - perf/mfu/actor:np.float64(0.03227397909516918) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.99407958984375) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006370724178850651 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006370724178850651 - critic/returns/max:0.875 - critic/returns/min:-0.875 -
(TaskRunner pid=2026458) Training Progress: 68%|██████▊ | 68/100 [47:50<18:09, 34.04s/it]
(TaskRunner pid=2026458) step:69 - global_seqlen/min:15072 - global_seqlen/max:24554 - global_seqlen/minmax_diff:9482 - global_seqlen/balanced_min:18606 - global_seqlen/balanced_max:18614 - global_seqlen/mean:18608.375 - actor/entropy:0.24838018417358398 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21835267543792725 - training/rollout_probs_diff_mean:0.004963765386492014 - training/rollout_probs_diff_std:0.012902619317173958 - training/rollout_actor_probs_pearson_corr:0.9984732270240784 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9996421337127686 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.750844502472319e-05 - rollout_corr/rollout_is_max:1.9659780263900757 - rollout_corr/rollout_is_min:0.26981014013290405 - rollout_corr/rollout_is_std:0.039976488798856735 - rollout_corr/rollout_is_eff_sample_size:0.9984033607144375 - rollout_corr/rollout_is_seq_mean:0.9996398687362671 - rollout_corr/rollout_is_seq_std:0.0026584318839013577 - rollout_corr/rollout_is_seq_max:1.0071197748184204 - rollout_corr/rollout_is_seq_min:0.9907388687133789 - rollout_corr/rollout_is_seq_max_deviation:0.009261131286621094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3142020958475769) - rollout_corr/training_log_ppl:np.float64(0.2681919914903119) - rollout_corr/kl:np.float64(0.0011005508938133346) - rollout_corr/k3_kl:np.float64(0.0010078800825397138) - rollout_corr/rollout_ppl:np.float64(1.3127118106931448) - rollout_corr/rollout_log_ppl:np.float64(0.2670914397749584) - rollout_corr/log_ppl_diff:np.float64(0.0011005517153535038) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023552981147076935) - rollout_corr/log_ppl_diff_max:np.float64(0.013139069080352783) - rollout_corr/log_ppl_diff_min:np.float64(-0.007561713457107544) - rollout_corr/ppl_ratio:np.float64(1.001105590723455) - rollout_corr/chi2_token:np.float64(0.0017845977563410997) - rollout_corr/chi2_seq:np.float64(0.37655540159903467) - actor/pg_loss:np.float64(0.004595633479766548) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005241842873147107) - actor/ppo_kl:np.float64(-0.0001359343073588093) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(242376.49935866683) - self_distillation/token_reweight_w_std:np.float64(3279716.472702818) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019234952051193) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08817204824299552) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.47684382647275925) - perf/mfu/actor:np.float64(0.03472723156176391) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.89558792114258) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034019839949905872 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034019839949905872 - critic/returns/max:0.875 - critic/returns/min:-0.875 - respo
(TaskRunner pid=2026458) Training Progress: 69%|██████▉ | 69/100 [48:20<16:58, 32.84s/it]
(TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70}
(TaskRunner pid=2026458) validation generation end
(TaskRunner pid=2026458) [prompt] system
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2026458) <reasoning>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) user
(TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2026458) Please reason step by step.
(TaskRunner pid=2026458) assistant
(TaskRunner pid=2026458) <think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) </think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) [response] <reasoning>
(TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". To determine the correct answer, we need to analyze the structure of the product and compare it with the options.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) The product contains a nitrile group (–n) attached to a cyclohexyl ring with a chlorine atom and a trifluoromethyl group (–C(F)(F)F). The structure also includes a benzene ring with a chlorine atom and a trifluoromethyl group.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Looking at the options:
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) - Option A: Contains a nitrile group and a cyclohexyl ring with a chlorine atom and a trifluoromethyl group, matching the product's structure.
(TaskRunner pid=2026458) - Option B: Contains a nitrile group but does not have the correct trifluoromethyl group or the correct chlorine atom placement.
(TaskRunner pid=2026458) - Option C: Contains a nitrile group but does not match the correct structure of the product, especially the trifluoromethyl group.
(TaskRunner pid=2026458) - Option D: Contains a nitrile group but does not have the correct structure of the product, especially the trifluoromethyl group.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it matches the structure of the product.
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458) [ground_truth] A
(TaskRunner pid=2026458) [score] 1.0
(TaskRunner pid=2026458) [acc] 1.0
(TaskRunner pid=2026458) [pred] A
(TaskRunner pid=2026458) [incorrect_format] 1
(TaskRunner pid=2026458) [feedback]
(TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70
(WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor
(TaskRunner pid=2026458) step:70 - global_seqlen/min:12837 - global_seqlen/max:19669 - global_seqlen/minmax_diff:6832 - global_seqlen/balanced_min:17821 - global_seqlen/balanced_max:17831 - global_seqlen/mean:17825.125 - actor/entropy:0.2367510199546814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33400535583496094 - training/rollout_probs_diff_mean:0.005260511301457882 - training/rollout_probs_diff_std:0.013495014049112797 - training/rollout_actor_probs_pearson_corr:0.9982122778892517 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.55859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.9999286532402039 - rollout_corr/rollout_is_ratio_fraction_high:4.527823330136016e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5092745343281422e-05 - rollout_corr/rollout_is_max:2.0651345252990723 - rollout_corr/rollout_is_min:0.28940898180007935 - rollout_corr/rollout_is_std:0.04225678741931915 - rollout_corr/rollout_is_eff_sample_size:0.9982174280795298 - rollout_corr/rollout_is_seq_mean:0.9999866485595703 - rollout_corr/rollout_is_seq_std:0.00303856679238379 - rollout_corr/rollout_is_seq_max:1.0101226568222046 - rollout_corr/rollout_is_seq_min:0.9901509284973145 - rollout_corr/rollout_is_seq_max_deviation:0.01012265682220459 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2921742992475629) - rollout_corr/training_log_ppl:np.float64(0.2528939112526132) - rollout_corr/kl:np.float64(0.0014884298633148774) - rollout_corr/k3_kl:np.float64(0.0010952503159273874) - rollout_corr/rollout_ppl:np.float64(1.2902372791431844) - rollout_corr/rollout_log_ppl:np.float64(0.2514054805797059) - rollout_corr/log_ppl_diff:np.float64(0.0014884306729072705) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028520045889308676) - rollout_corr/log_ppl_diff_max:np.float64(0.0136488676071167) - rollout_corr/log_ppl_diff_min:np.float64(-0.008753955364227295) - rollout_corr/ppl_ratio:np.float64(1.0014950232580304) - rollout_corr/chi2_token:np.float64(0.0014098831452429295) - rollout_corr/chi2_seq:np.float64(0.360071137547493) - actor/pg_loss:np.float64(0.0027717185439541936) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003434370364630013) - actor/ppo_kl:np.float64(0.0005119445677905787) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_w_mean:np.float64(69071.25273821992) - self_distillation/token_reweight_w_std:np.float64(1045658.8316577639) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004556903149933) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06493389664683491) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.3887019380927086) - perf/mfu/actor:np.float64(0.0325070599234879) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.87537002563477) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.130425
(TaskRunner pid=2026458) Training Progress: 70%|███████ | 70/100 [50:19<29:18, 58.61s/it]
(TaskRunner pid=2026458) step:71 - global_seqlen/min:15483 - global_seqlen/max:28966 - global_seqlen/minmax_diff:13483 - global_seqlen/balanced_min:18380 - global_seqlen/balanced_max:25891 - global_seqlen/mean:19333.5 - actor/entropy:0.21173985302448273 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4046512842178345 - training/rollout_probs_diff_mean:0.0045140874572098255 - training/rollout_probs_diff_std:0.012615382671356201 - training/rollout_actor_probs_pearson_corr:0.9983184337615967 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9998964667320251 - rollout_corr/rollout_is_ratio_fraction_high:1.2943308320245706e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.471654342021793e-05 - rollout_corr/rollout_is_max:2.0498809814453125 - rollout_corr/rollout_is_min:0.38866347074508667 - rollout_corr/rollout_is_std:0.03843024745583534 - rollout_corr/rollout_is_eff_sample_size:0.9985250563653204 - rollout_corr/rollout_is_seq_mean:0.9998904466629028 - rollout_corr/rollout_is_seq_std:0.0028965510427951813 - rollout_corr/rollout_is_seq_max:1.0107746124267578 - rollout_corr/rollout_is_seq_min:0.9922298192977905 - rollout_corr/rollout_is_seq_max_deviation:0.010774612426757812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2945193266496062) - rollout_corr/training_log_ppl:np.float64(0.2542406297643538) - rollout_corr/kl:np.float64(0.0010824492369110317) - rollout_corr/k3_kl:np.float64(0.0011051825894128342) - rollout_corr/rollout_ppl:np.float64(1.293056899216026) - rollout_corr/rollout_log_ppl:np.float64(0.2531581802359142) - rollout_corr/log_ppl_diff:np.float64(0.0010824495284396107) - rollout_corr/log_ppl_abs_diff:np.float64(0.00271785432141769) - rollout_corr/log_ppl_diff_max:np.float64(0.014131128787994385) - rollout_corr/log_ppl_diff_min:np.float64(-0.009531468152999878) - rollout_corr/ppl_ratio:np.float64(1.0010888520628214) - rollout_corr/chi2_token:np.float64(0.002212358172982931) - rollout_corr/chi2_seq:np.float64(0.6793141439557076) - actor/pg_loss:np.float64(0.002975177369080484) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000972424295014207) - actor/ppo_kl:np.float64(7.461081707615946e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(134136.12361396686) - self_distillation/token_reweight_w_std:np.float64(1624645.7259825736) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988281566184014) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07092063533491455) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.4537533223628998) - perf/mfu/actor:np.float64(0.03374089352858298) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.65079879760742) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06832286715507507 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06832286715507507 - critic/returns/max:0.875 - critic/returns/min:-0.875 - respo
(TaskRunner pid=2026458) Training Progress: 71%|███████ | 71/100 [51:32<30:23, 62.89s/it]
(TaskRunner pid=2026458) step:72 - global_seqlen/min:14180 - global_seqlen/max:19404 - global_seqlen/minmax_diff:5224 - global_seqlen/balanced_min:16784 - global_seqlen/balanced_max:16786 - global_seqlen/mean:16785.25 - actor/entropy:0.2532234191894531 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37658625841140747 - training/rollout_probs_diff_mean:0.005358405876904726 - training/rollout_probs_diff_std:0.01324445940554142 - training/rollout_actor_probs_pearson_corr:0.9983224272727966 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.000357747077942 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6467605829238892 - rollout_corr/rollout_is_min:0.5746954679489136 - rollout_corr/rollout_is_std:0.04072979837656021 - rollout_corr/rollout_is_eff_sample_size:0.9983450191216581 - rollout_corr/rollout_is_seq_mean:1.0003376007080078 - rollout_corr/rollout_is_seq_std:0.0029273421969264746 - rollout_corr/rollout_is_seq_max:1.0102252960205078 - rollout_corr/rollout_is_seq_min:0.9919271469116211 - rollout_corr/rollout_is_seq_max_deviation:0.010225296020507812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3001699862070382) - rollout_corr/training_log_ppl:np.float64(0.25781238308991306) - rollout_corr/kl:np.float64(0.0006664992815785808) - rollout_corr/k3_kl:np.float64(0.0009609586412011595) - rollout_corr/rollout_ppl:np.float64(1.2992715733125806) - rollout_corr/rollout_log_ppl:np.float64(0.25714588271512184) - rollout_corr/log_ppl_diff:np.float64(0.0006665003747912124) - rollout_corr/log_ppl_abs_diff:np.float64(0.002382488935836591) - rollout_corr/log_ppl_diff_max:np.float64(0.009035348892211914) - rollout_corr/log_ppl_diff_min:np.float64(-0.009852290153503418) - rollout_corr/ppl_ratio:np.float64(1.0006713322363794) - rollout_corr/chi2_token:np.float64(0.0025634136982262135) - rollout_corr/chi2_seq:np.float64(1.15624994575046) - actor/pg_loss:np.float64(0.003654895001091063) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009709688329166966) - actor/ppo_kl:np.float64(0.00014433233151223135) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(213152.86371882842) - self_distillation/token_reweight_w_std:np.float64(2646677.052918058) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9974161058198661) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09049051211331971) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.43892450630664825) - perf/mfu/actor:np.float64(0.03149296183920366) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.40920639038086) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009050944820046425 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009050944820046425 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:237.6953125
(TaskRunner pid=2026458) Training Progress: 72%|███████▏ | 72/100 [52:00<24:27, 52.41s/it]
(TaskRunner pid=2026458) step:73 - global_seqlen/min:15559 - global_seqlen/max:20190 - global_seqlen/minmax_diff:4631 - global_seqlen/balanced_min:17173 - global_seqlen/balanced_max:17174 - global_seqlen/mean:17173.5 - actor/entropy:0.24391600489616394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135395884513855 - training/rollout_probs_diff_mean:0.0049339784309268 - training/rollout_probs_diff_std:0.012776896357536316 - training/rollout_actor_probs_pearson_corr:0.998493492603302 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.000167965888977 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.0463656003121287e-05 - rollout_corr/rollout_is_max:1.767087697982788 - rollout_corr/rollout_is_min:0.47380420565605164 - rollout_corr/rollout_is_std:0.03922998905181885 - rollout_corr/rollout_is_eff_sample_size:0.9984639671449654 - rollout_corr/rollout_is_seq_mean:1.0002386569976807 - rollout_corr/rollout_is_seq_std:0.0029904802795499563 - rollout_corr/rollout_is_seq_max:1.0164623260498047 - rollout_corr/rollout_is_seq_min:0.9890998005867004 - rollout_corr/rollout_is_seq_max_deviation:0.016462326049804688 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2968735643662512) - rollout_corr/training_log_ppl:np.float64(0.25511286556138657) - rollout_corr/kl:np.float64(0.0009459122673263209) - rollout_corr/k3_kl:np.float64(0.0009600993125502555) - rollout_corr/rollout_ppl:np.float64(1.2955679851584136) - rollout_corr/rollout_log_ppl:np.float64(0.2541669542551972) - rollout_corr/log_ppl_diff:np.float64(0.0009459113061893731) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024720369547139853) - rollout_corr/log_ppl_diff_max:np.float64(0.012520670890808105) - rollout_corr/log_ppl_diff_min:np.float64(-0.0084361732006073) - rollout_corr/ppl_ratio:np.float64(1.0009514002595097) - rollout_corr/chi2_token:np.float64(0.001954789971932769) - rollout_corr/chi2_seq:np.float64(0.6583140406291932) - actor/pg_loss:np.float64(0.0032611776841804385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000914203752472531) - actor/ppo_kl:np.float64(0.00038008014968227144) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.546875) - self_distillation/token_reweight_w_mean:np.float64(311191.56539016124) - self_distillation/token_reweight_w_std:np.float64(2860058.778589467) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015116163995117) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06564340484328568) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.42411575466394424) - perf/mfu/actor:np.float64(0.031904336867600726) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.16930770874023) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001247106003575027 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001247106003575027 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:256.
(TaskRunner pid=2026458) Training Progress: 73%|███████▎ | 73/100 [52:28<20:17, 45.09s/it]
(TaskRunner pid=2026458) step:74 - global_seqlen/min:13447 - global_seqlen/max:21573 - global_seqlen/minmax_diff:8126 - global_seqlen/balanced_min:16831 - global_seqlen/balanced_max:16917 - global_seqlen/mean:16843.875 - actor/entropy:0.24996380507946014 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424726128578186 - training/rollout_probs_diff_mean:0.004949183203279972 - training/rollout_probs_diff_std:0.012561521492898464 - training/rollout_actor_probs_pearson_corr:0.9985408782958984 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9998319149017334 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.6325607399921864e-05 - rollout_corr/rollout_is_max:1.7145717144012451 - rollout_corr/rollout_is_min:0.17540410161018372 - rollout_corr/rollout_is_std:0.03882066160440445 - rollout_corr/rollout_is_eff_sample_size:0.9984948673727597 - rollout_corr/rollout_is_seq_mean:0.9999223947525024 - rollout_corr/rollout_is_seq_std:0.0029323359485715628 - rollout_corr/rollout_is_seq_max:1.009851336479187 - rollout_corr/rollout_is_seq_min:0.9908139109611511 - rollout_corr/rollout_is_seq_max_deviation:0.009851336479187012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3159835655242205) - rollout_corr/training_log_ppl:np.float64(0.26891483918007) - rollout_corr/kl:np.float64(0.0010735450956096404) - rollout_corr/k3_kl:np.float64(0.0011378318484958072) - rollout_corr/rollout_ppl:np.float64(1.3145075496286154) - rollout_corr/rollout_log_ppl:np.float64(0.26784129165753257) - rollout_corr/log_ppl_diff:np.float64(0.0010735475225374103) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030432986677624285) - rollout_corr/log_ppl_diff_max:np.float64(0.012433648109436035) - rollout_corr/log_ppl_diff_min:np.float64(-0.009890466928482056) - rollout_corr/ppl_ratio:np.float64(1.0010810112580657) - rollout_corr/chi2_token:np.float64(0.0024086034391075373) - rollout_corr/chi2_seq:np.float64(1.130532833514735) - actor/pg_loss:np.float64(0.0028035463765263557) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007331344913836801) - actor/ppo_kl:np.float64(0.00016281594710676472) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(183593.82668150146) - self_distillation/token_reweight_w_std:np.float64(2093931.331328835) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001840919721872) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060318807751173154) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.3842148035764694) - perf/mfu/actor:np.float64(0.031244242735268143) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2487654685974) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0016619311645627022 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0016619311645627022 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_leng
(TaskRunner pid=2026458) Training Progress: 74%|███████▍ | 74/100 [52:58<17:37, 40.66s/it]
(TaskRunner pid=2026458) step:75 - global_seqlen/min:14038 - global_seqlen/max:20039 - global_seqlen/minmax_diff:6001 - global_seqlen/balanced_min:18453 - global_seqlen/balanced_max:18459 - global_seqlen/mean:18457.625 - actor/entropy:0.24194937944412231 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4665491580963135 - training/rollout_probs_diff_mean:0.005109612364321947 - training/rollout_probs_diff_std:0.013375498354434967 - training/rollout_actor_probs_pearson_corr:0.9982732534408569 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9997668862342834 - rollout_corr/rollout_is_ratio_fraction_high:2.9664348403457552e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.449652260518633e-05 - rollout_corr/rollout_is_max:2.278925657272339 - rollout_corr/rollout_is_min:0.3029129207134247 - rollout_corr/rollout_is_std:0.040616225451231 - rollout_corr/rollout_is_eff_sample_size:0.9983523857083623 - rollout_corr/rollout_is_seq_mean:0.9997424483299255 - rollout_corr/rollout_is_seq_std:0.002785229589790106 - rollout_corr/rollout_is_seq_max:1.00969660282135 - rollout_corr/rollout_is_seq_min:0.9920895099639893 - rollout_corr/rollout_is_seq_max_deviation:0.009696602821350098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2928350591100752) - rollout_corr/training_log_ppl:np.float64(0.2525325442838948) - rollout_corr/kl:np.float64(0.0011320576819571215) - rollout_corr/k3_kl:np.float64(0.0010110275231340893) - rollout_corr/rollout_ppl:np.float64(1.2913245973177254) - rollout_corr/rollout_log_ppl:np.float64(0.2514004855765961) - rollout_corr/log_ppl_diff:np.float64(0.001132058707298711) - rollout_corr/log_ppl_abs_diff:np.float64(0.002472092892276123) - rollout_corr/log_ppl_diff_max:np.float64(0.01102399080991745) - rollout_corr/log_ppl_diff_min:np.float64(-0.008037805557250977) - rollout_corr/ppl_ratio:np.float64(1.0011374710593373) - rollout_corr/chi2_token:np.float64(0.0017729024402797222) - rollout_corr/chi2_seq:np.float64(0.6597908593248576) - actor/pg_loss:np.float64(0.0018322557443752885) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036231086232874077) - actor/ppo_kl:np.float64(-3.7132614664869834e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(138490.71220888244) - self_distillation/token_reweight_w_std:np.float64(1204680.9713628318) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000044918153435) - self_distillation/token_reweight_w_clip_frac:np.float64(0.041466833936283365) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.29594044387340546) - perf/mfu/actor:np.float64(0.03412545800281451) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2256622314453) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:6.489075894933194e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:6.489075894933194e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean
(TaskRunner pid=2026458) Training Progress: 75%|███████▌ | 75/100 [53:28<15:33, 37.33s/it]
(TaskRunner pid=2026458) step:76 - global_seqlen/min:15400 - global_seqlen/max:20933 - global_seqlen/minmax_diff:5533 - global_seqlen/balanced_min:18318 - global_seqlen/balanced_max:18443 - global_seqlen/mean:18349.25 - actor/entropy:0.23565568029880524 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6839720010757446 - training/rollout_probs_diff_mean:0.00476441252976656 - training/rollout_probs_diff_std:0.01295106578618288 - training/rollout_actor_probs_pearson_corr:0.9984387159347534 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:0.999829113483429 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.072877345606685e-05 - rollout_corr/rollout_is_max:1.8591279983520508 - rollout_corr/rollout_is_min:0.14236801862716675 - rollout_corr/rollout_is_std:0.038981545716524124 - rollout_corr/rollout_is_eff_sample_size:0.9984822693566601 - rollout_corr/rollout_is_seq_mean:0.9998883008956909 - rollout_corr/rollout_is_seq_std:0.0026900693774223328 - rollout_corr/rollout_is_seq_max:1.0127232074737549 - rollout_corr/rollout_is_seq_min:0.9910237193107605 - rollout_corr/rollout_is_seq_max_deviation:0.012723207473754883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3050027494318783) - rollout_corr/training_log_ppl:np.float64(0.26096004332066514) - rollout_corr/kl:np.float64(0.001120272666678801) - rollout_corr/k3_kl:np.float64(0.000998899616035942) - rollout_corr/rollout_ppl:np.float64(1.3034690767526627) - rollout_corr/rollout_log_ppl:np.float64(0.25983976959832944) - rollout_corr/log_ppl_diff:np.float64(0.0011202737223356962) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024523677420802414) - rollout_corr/log_ppl_diff_max:np.float64(0.01079338788986206) - rollout_corr/log_ppl_diff_min:np.float64(-0.011696979403495789) - rollout_corr/ppl_ratio:np.float64(1.001125503797084) - rollout_corr/chi2_token:np.float64(0.0016837881412357092) - rollout_corr/chi2_seq:np.float64(0.4817654227372259) - actor/pg_loss:np.float64(0.0022594364127144217) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005095393812553084) - actor/ppo_kl:np.float64(0.000131376795138749) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(86914.96509032976) - self_distillation/token_reweight_w_std:np.float64(1369863.7220702441) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999393313191831) - self_distillation/token_reweight_w_clip_frac:np.float64(0.041579058073693886) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.3407774642109871) - perf/mfu/actor:np.float64(0.034170829781458016) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.09953689575195) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002358874771744013 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002358874771744013 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:287.7265625 - response_len
(TaskRunner pid=2026458) Training Progress: 76%|███████▌ | 76/100 [53:59<14:09, 35.40s/it]
(TaskRunner pid=2026458) step:77 - global_seqlen/min:16068 - global_seqlen/max:21136 - global_seqlen/minmax_diff:5068 - global_seqlen/balanced_min:18347 - global_seqlen/balanced_max:18355 - global_seqlen/mean:18353.125 - actor/entropy:0.24358664453029633 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3763711750507355 - training/rollout_probs_diff_mean:0.005085653159767389 - training/rollout_probs_diff_std:0.012961719185113907 - training/rollout_actor_probs_pearson_corr:0.9984139204025269 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999815821647644 - rollout_corr/rollout_is_ratio_fraction_high:1.5232524674502201e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.61626215535216e-05 - rollout_corr/rollout_is_max:2.443542003631592 - rollout_corr/rollout_is_min:0.0030977046117186546 - rollout_corr/rollout_is_std:0.04062869772315025 - rollout_corr/rollout_is_eff_sample_size:0.9983521480748362 - rollout_corr/rollout_is_seq_mean:1.0000320672988892 - rollout_corr/rollout_is_seq_std:0.0030800357926636934 - rollout_corr/rollout_is_seq_max:1.008837342262268 - rollout_corr/rollout_is_seq_min:0.9924508333206177 - rollout_corr/rollout_is_seq_max_deviation:0.008837342262268066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.30547040887177) - rollout_corr/training_log_ppl:np.float64(0.262247261358425) - rollout_corr/kl:np.float64(0.001232846368517393) - rollout_corr/k3_kl:np.float64(0.0012621798805412254) - rollout_corr/rollout_ppl:np.float64(1.303758448921144) - rollout_corr/rollout_log_ppl:np.float64(0.2610144141508499) - rollout_corr/log_ppl_diff:np.float64(0.0012328472075751051) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028479194588726386) - rollout_corr/log_ppl_diff_max:np.float64(0.01525259017944336) - rollout_corr/log_ppl_diff_min:np.float64(-0.007772982120513916) - rollout_corr/ppl_ratio:np.float64(1.0012400422710925) - rollout_corr/chi2_token:np.float64(0.0023893536999821663) - rollout_corr/chi2_seq:np.float64(0.8159184374380857) - actor/pg_loss:np.float64(0.002790838829241693) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009325358823843999) - actor/ppo_kl:np.float64(0.0003359223030457992) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(180959.5167653456) - self_distillation/token_reweight_w_std:np.float64(2185127.907001757) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988750158809125) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06124208416440524) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.4322587177157402) - perf/mfu/actor:np.float64(0.034241717008689325) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1767692565918) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0021534981206059456 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0021534981206059456 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:256.44
(TaskRunner pid=2026458) Training Progress: 77%|███████▋ | 77/100 [54:28<12:55, 33.71s/it]
(TaskRunner pid=2026458) step:78 - global_seqlen/min:15356 - global_seqlen/max:19696 - global_seqlen/minmax_diff:4340 - global_seqlen/balanced_min:17649 - global_seqlen/balanced_max:17655 - global_seqlen/mean:17652.75 - actor/entropy:0.23699037730693817 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975830912590027 - training/rollout_probs_diff_mean:0.005081570707261562 - training/rollout_probs_diff_std:0.013243689201772213 - training/rollout_actor_probs_pearson_corr:0.9983371496200562 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999217987060547 - rollout_corr/rollout_is_ratio_fraction_high:4.307003109715879e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.307003109715879e-05 - rollout_corr/rollout_is_max:2.9357402324676514 - rollout_corr/rollout_is_min:0.4458056390285492 - rollout_corr/rollout_is_std:0.04097927734255791 - rollout_corr/rollout_is_eff_sample_size:0.9983230388235971 - rollout_corr/rollout_is_seq_mean:1.0000085830688477 - rollout_corr/rollout_is_seq_std:0.002981689525768161 - rollout_corr/rollout_is_seq_max:1.0086852312088013 - rollout_corr/rollout_is_seq_min:0.9888802170753479 - rollout_corr/rollout_is_seq_max_deviation:0.0111197829246521 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3004110623151064) - rollout_corr/training_log_ppl:np.float64(0.257169630160206) - rollout_corr/kl:np.float64(0.0012380461494814199) - rollout_corr/k3_kl:np.float64(0.001083872083654569) - rollout_corr/rollout_ppl:np.float64(1.2987679266370833) - rollout_corr/rollout_log_ppl:np.float64(0.25593158273841254) - rollout_corr/log_ppl_diff:np.float64(0.0012380474217934534) - rollout_corr/log_ppl_abs_diff:np.float64(0.002574279482359998) - rollout_corr/log_ppl_diff_max:np.float64(0.015157654881477356) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076891034841537476) - rollout_corr/ppl_ratio:np.float64(1.0012440220452845) - rollout_corr/chi2_token:np.float64(0.001788182882592082) - rollout_corr/chi2_seq:np.float64(1.9581220541149378) - actor/pg_loss:np.float64(0.0033810948953032494) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009038109546963824) - actor/ppo_kl:np.float64(0.00033419562218028886) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(189436.75045557343) - self_distillation/token_reweight_w_std:np.float64(2503367.794994272) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009928282815963) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07160566429956816) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3780764937400818) - perf/mfu/actor:np.float64(0.03291894268049457) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.0838165283203) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019830160308629274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019830160308629274 - critic/returns/max:0.875 - critic/returns/min:-0.875
(TaskRunner pid=2026458) Training Progress: 78%|███████▊ | 78/100 [54:57<11:51, 32.32s/it]
(TaskRunner pid=2026458) step:79 - global_seqlen/min:13318 - global_seqlen/max:19723 - global_seqlen/minmax_diff:6405 - global_seqlen/balanced_min:16851 - global_seqlen/balanced_max:16855 - global_seqlen/mean:16852.75 - actor/entropy:0.2380763292312622 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24131831526756287 - training/rollout_probs_diff_mean:0.00509747164323926 - training/rollout_probs_diff_std:0.01309633906930685 - training/rollout_actor_probs_pearson_corr:0.998324453830719 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.999850869178772 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4865467164781876e-05 - rollout_corr/rollout_is_max:1.6486144065856934 - rollout_corr/rollout_is_min:0.48596420884132385 - rollout_corr/rollout_is_std:0.04019506648182869 - rollout_corr/rollout_is_eff_sample_size:0.998386606117186 - rollout_corr/rollout_is_seq_mean:0.9999116659164429 - rollout_corr/rollout_is_seq_std:0.0028780419379472733 - rollout_corr/rollout_is_seq_max:1.0085704326629639 - rollout_corr/rollout_is_seq_min:0.9908472299575806 - rollout_corr/rollout_is_seq_max_deviation:0.009152770042419434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2909451411105692) - rollout_corr/training_log_ppl:np.float64(0.250693125941325) - rollout_corr/kl:np.float64(0.0009612457183649781) - rollout_corr/k3_kl:np.float64(0.0010165603632685816) - rollout_corr/rollout_ppl:np.float64(1.2896752119995654) - rollout_corr/rollout_log_ppl:np.float64(0.24973187799332663) - rollout_corr/log_ppl_diff:np.float64(0.0009612479479983449) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024955651024356484) - rollout_corr/log_ppl_diff_max:np.float64(0.008482903242111206) - rollout_corr/log_ppl_diff_min:np.float64(-0.007829368114471436) - rollout_corr/ppl_ratio:np.float64(1.0009662464726716) - rollout_corr/chi2_token:np.float64(0.002142100827768445) - rollout_corr/chi2_seq:np.float64(0.36312326532788575) - actor/pg_loss:np.float64(0.0036202146438881755) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008493167902088317) - actor/ppo_kl:np.float64(1.813888876256442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(207000.82824485935) - self_distillation/token_reweight_w_std:np.float64(2774328.0958690653) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004315504338592) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07667552542989142) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.5015818998217583) - perf/mfu/actor:np.float64(0.031159245026246143) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.08302688598633) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000724691548384726 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000724691548384726 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:262.77343
(TaskRunner pid=2026458) Training Progress: 79%|███████▉ | 79/100 [55:27<11:02, 31.52s/it]
(TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80}
(TaskRunner pid=2026458) validation generation end
(TaskRunner pid=2026458) [prompt] system
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2026458) <reasoning>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) user
(TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2026458) Please reason step by step.
(TaskRunner pid=2026458) assistant
(TaskRunner pid=2026458) <think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) </think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) [response] <reasoning>
(TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a nitrile group (n1) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group (C(F)(F)F).
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains a nitrile group (n1) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group, matching the product structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option B: "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" contains a chlorine atom but does not have the correct nitrile group and trifluoromethyl group structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option C: "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" has a different structure with an oxygen atom and a different substitution pattern, not matching the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option D: "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" contains a different structure with a double bond and a different substitution pattern, not matching the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it matches the structure of the product.
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458) [ground_truth] A
(TaskRunner pid=2026458) [score] 1.0
(TaskRunner pid=2026458) [acc] 1.0
(TaskRunner pid=2026458) [pred] A
(TaskRunner pid=2026458) [incorrect_format] 1
(TaskRunner pid=2026458) [feedback]
(TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80
(WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor
(TaskRunner pid=2026458) step:80 - global_seqlen/min:16122 - global_seqlen/max:23121 - global_seqlen/minmax_diff:6999 - global_seqlen/balanced_min:18851 - global_seqlen/balanced_max:18860 - global_seqlen/mean:18857.375 - actor/entropy:0.22489960491657257 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3807961344718933 - training/rollout_probs_diff_mean:0.005050069652497768 - training/rollout_probs_diff_std:0.013410150073468685 - training/rollout_actor_probs_pearson_corr:0.998170018196106 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0001280307769775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.252001963322982e-05 - rollout_corr/rollout_is_max:1.815222144126892 - rollout_corr/rollout_is_min:0.23840588331222534 - rollout_corr/rollout_is_std:0.040402136743068695 - rollout_corr/rollout_is_eff_sample_size:0.9983706838295904 - rollout_corr/rollout_is_seq_mean:1.0001842975616455 - rollout_corr/rollout_is_seq_std:0.002584717469289899 - rollout_corr/rollout_is_seq_max:1.0079708099365234 - rollout_corr/rollout_is_seq_min:0.9935450553894043 - rollout_corr/rollout_is_seq_max_deviation:0.007970809936523438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.284651028458029) - rollout_corr/training_log_ppl:np.float64(0.2446684459282551) - rollout_corr/kl:np.float64(0.0011069629296169836) - rollout_corr/k3_kl:np.float64(0.0011388705046897485) - rollout_corr/rollout_ppl:np.float64(1.2831353461369872) - rollout_corr/rollout_log_ppl:np.float64(0.24356148090737406) - rollout_corr/log_ppl_diff:np.float64(0.0011069650208810344) - rollout_corr/log_ppl_abs_diff:np.float64(0.002547982890973799) - rollout_corr/log_ppl_diff_max:np.float64(0.011913180351257324) - rollout_corr/log_ppl_diff_min:np.float64(-0.00954301655292511) - rollout_corr/ppl_ratio:np.float64(1.0011127449106425) - rollout_corr/chi2_token:np.float64(0.0022547231055796146) - rollout_corr/chi2_seq:np.float64(0.705732943257317) - actor/pg_loss:np.float64(0.003671735990792513) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012357970999801182) - actor/ppo_kl:np.float64(0.00041334695177397407) - actor/pg_clipfrac_lower:np.float64(2.100134406646248e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(147860.0034218824) - self_distillation/token_reweight_w_std:np.float64(2243142.9639211292) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017643314786255) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08743216533912346) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.46084368228912354) - perf/mfu/actor:np.float64(0.0348035546181388) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.0935287475586) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.124849094228493
(TaskRunner pid=2026458) Training Progress: 80%|████████ | 80/100 [57:16<18:12, 54.65s/it]
(TaskRunner pid=2026458) step:81 - global_seqlen/min:15749 - global_seqlen/max:23070 - global_seqlen/minmax_diff:7321 - global_seqlen/balanced_min:19345 - global_seqlen/balanced_max:19350 - global_seqlen/mean:19349.125 - actor/entropy:0.24268725514411926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.0049947695806622505 - training/rollout_probs_diff_std:0.013053881004452705 - training/rollout_actor_probs_pearson_corr:0.9983764290809631 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9997670650482178 - rollout_corr/rollout_is_ratio_fraction_high:1.3156336535757873e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.262534614303149e-05 - rollout_corr/rollout_is_max:2.0260822772979736 - rollout_corr/rollout_is_min:0.2609400451183319 - rollout_corr/rollout_is_std:0.03953344747424126 - rollout_corr/rollout_is_eff_sample_size:0.9984388918274393 - rollout_corr/rollout_is_seq_mean:0.999757707118988 - rollout_corr/rollout_is_seq_std:0.002598333638161421 - rollout_corr/rollout_is_seq_max:1.0065761804580688 - rollout_corr/rollout_is_seq_min:0.9917704463005066 - rollout_corr/rollout_is_seq_max_deviation:0.008229553699493408 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2969164904206991) - rollout_corr/training_log_ppl:np.float64(0.2555096454452723) - rollout_corr/kl:np.float64(0.0012044042446035519) - rollout_corr/k3_kl:np.float64(0.001068380127094315) - rollout_corr/rollout_ppl:np.float64(1.295284100342542) - rollout_corr/rollout_log_ppl:np.float64(0.25430524366674945) - rollout_corr/log_ppl_diff:np.float64(0.0012044017785228789) - rollout_corr/log_ppl_abs_diff:np.float64(0.002569538773968816) - rollout_corr/log_ppl_diff_max:np.float64(0.00988227128982544) - rollout_corr/log_ppl_diff_min:np.float64(-0.007966101169586182) - rollout_corr/ppl_ratio:np.float64(1.0012098925653845) - rollout_corr/chi2_token:np.float64(0.0018374514766037464) - rollout_corr/chi2_seq:np.float64(0.6602487270720303) - actor/pg_loss:np.float64(0.0036679430631920695) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007375691257038852) - actor/ppo_kl:np.float64(0.00012186931077984298) - actor/pg_clipfrac_lower:np.float64(8.037551197048742e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(152346.46296780813) - self_distillation/token_reweight_w_std:np.float64(2223816.070475471) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006990339607) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07411914720432833) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.43313899636268616) - perf/mfu/actor:np.float64(0.035338759221170066) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.7124252319336) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0049204700626432896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0049204700626432896 - critic/returns/max:0.875 - critic/returns
(TaskRunner pid=2026458) Training Progress: 81%|████████ | 81/100 [57:46<15:01, 47.43s/it]
(TaskRunner pid=2026458) step:82 - global_seqlen/min:15004 - global_seqlen/max:21121 - global_seqlen/minmax_diff:6117 - global_seqlen/balanced_min:17893 - global_seqlen/balanced_max:17896 - global_seqlen/mean:17895.25 - actor/entropy:0.23795437812805176 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31782522797584534 - training/rollout_probs_diff_mean:0.004893430043011904 - training/rollout_probs_diff_std:0.012663507834076881 - training/rollout_actor_probs_pearson_corr:0.9984432458877563 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.000130534172058 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.6648900529835373e-05 - rollout_corr/rollout_is_max:1.7479747533798218 - rollout_corr/rollout_is_min:0.3629961311817169 - rollout_corr/rollout_is_std:0.038970839232206345 - rollout_corr/rollout_is_eff_sample_size:0.998484170923625 - rollout_corr/rollout_is_seq_mean:1.0002964735031128 - rollout_corr/rollout_is_seq_std:0.002696148119866848 - rollout_corr/rollout_is_seq_max:1.0123590230941772 - rollout_corr/rollout_is_seq_min:0.9923802018165588 - rollout_corr/rollout_is_seq_max_deviation:0.012359023094177246 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.296073100529611) - rollout_corr/training_log_ppl:np.float64(0.2552825820748694) - rollout_corr/kl:np.float64(0.0009574908228102696) - rollout_corr/k3_kl:np.float64(0.0011017269548574404) - rollout_corr/rollout_ppl:np.float64(1.2947798878885806) - rollout_corr/rollout_log_ppl:np.float64(0.254325088521) - rollout_corr/log_ppl_diff:np.float64(0.0009574935538694263) - rollout_corr/log_ppl_abs_diff:np.float64(0.002439796517137438) - rollout_corr/log_ppl_diff_max:np.float64(0.014934465289115906) - rollout_corr/log_ppl_diff_min:np.float64(-0.010826826095581055) - rollout_corr/ppl_ratio:np.float64(1.0009630019776523) - rollout_corr/chi2_token:np.float64(0.002452905522659421) - rollout_corr/chi2_seq:np.float64(0.423899469897151) - actor/pg_loss:np.float64(0.0041424884693697095) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013430351091301418) - actor/ppo_kl:np.float64(0.000410171516927349) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(233069.21662305342) - self_distillation/token_reweight_w_std:np.float64(2612623.9466522) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009098192676902) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0835862404492218) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.45514093339443207) - perf/mfu/actor:np.float64(0.03317730103506178) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.53085327148438) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003221185877919197 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003221185877919197 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.1640625 - response_le
(TaskRunner pid=2026458) Training Progress: 82%|████████▏ | 82/100 [58:17<12:42, 42.36s/it]
(TaskRunner pid=2026458) step:83 - global_seqlen/min:15510 - global_seqlen/max:20545 - global_seqlen/minmax_diff:5035 - global_seqlen/balanced_min:18663 - global_seqlen/balanced_max:18665 - global_seqlen/mean:18664.0 - actor/entropy:0.253889262676239 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553972005844116 - training/rollout_probs_diff_mean:0.005415121093392372 - training/rollout_probs_diff_std:0.013402428478002548 - training/rollout_actor_probs_pearson_corr:0.9983406662940979 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:1.0002059936523438 - rollout_corr/rollout_is_ratio_fraction_high:2.7475547540234402e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3737773770117201e-05 - rollout_corr/rollout_is_max:2.8205859661102295 - rollout_corr/rollout_is_min:0.3914983570575714 - rollout_corr/rollout_is_std:0.04187990352511406 - rollout_corr/rollout_is_eff_sample_size:0.9982497385555893 - rollout_corr/rollout_is_seq_mean:1.000227689743042 - rollout_corr/rollout_is_seq_std:0.002809274010360241 - rollout_corr/rollout_is_seq_max:1.009268045425415 - rollout_corr/rollout_is_seq_min:0.9934084415435791 - rollout_corr/rollout_is_seq_max_deviation:0.009268045425415039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3143565426580608) - rollout_corr/training_log_ppl:np.float64(0.2695897765806876) - rollout_corr/kl:np.float64(0.000838180147326284) - rollout_corr/k3_kl:np.float64(0.001138548173230447) - rollout_corr/rollout_ppl:np.float64(1.313231692649424) - rollout_corr/rollout_log_ppl:np.float64(0.26875159548944794) - rollout_corr/log_ppl_diff:np.float64(0.0008381810912396759) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024882252619136125) - rollout_corr/log_ppl_diff_max:np.float64(0.013383328914642334) - rollout_corr/log_ppl_diff_min:np.float64(-0.00783604383468628) - rollout_corr/ppl_ratio:np.float64(1.0008434464689344) - rollout_corr/chi2_token:np.float64(0.002809858415275812) - rollout_corr/chi2_seq:np.float64(0.9911151258274913) - actor/pg_loss:np.float64(0.004593900288455188) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000947352435105131) - actor/ppo_kl:np.float64(0.00014004049228333315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(142169.04989654478) - self_distillation/token_reweight_w_std:np.float64(1783576.419247636) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992821563500911) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07421927264658734) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.4911562651395798) - perf/mfu/actor:np.float64(0.03439597980816678) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.20166397094727) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015317617217078805 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015317617217078805 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:284.34375 - respons
(TaskRunner pid=2026458) Training Progress: 83%|████████▎ | 83/100 [58:47<10:58, 38.73s/it]
(TaskRunner pid=2026458) step:84 - global_seqlen/min:14717 - global_seqlen/max:21368 - global_seqlen/minmax_diff:6651 - global_seqlen/balanced_min:17413 - global_seqlen/balanced_max:17422 - global_seqlen/mean:17419.375 - actor/entropy:0.22988727688789368 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35856932401657104 - training/rollout_probs_diff_mean:0.005215250421315432 - training/rollout_probs_diff_std:0.013718393631279469 - training/rollout_actor_probs_pearson_corr:0.9981431365013123 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9996470212936401 - rollout_corr/rollout_is_ratio_fraction_high:1.3454785403155256e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.072871423792094e-05 - rollout_corr/rollout_is_max:2.1147913932800293 - rollout_corr/rollout_is_min:0.390633225440979 - rollout_corr/rollout_is_std:0.041462238878011703 - rollout_corr/rollout_is_eff_sample_size:0.9982825263238067 - rollout_corr/rollout_is_seq_mean:0.9997439980506897 - rollout_corr/rollout_is_seq_std:0.0025515363086014986 - rollout_corr/rollout_is_seq_max:1.0085729360580444 - rollout_corr/rollout_is_seq_min:0.9917205572128296 - rollout_corr/rollout_is_seq_max_deviation:0.008572936058044434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2788517312146723) - rollout_corr/training_log_ppl:np.float64(0.24251381697831675) - rollout_corr/kl:np.float64(0.0013615037188472456) - rollout_corr/k3_kl:np.float64(0.0013387326044949077) - rollout_corr/rollout_ppl:np.float64(1.2770492630079389) - rollout_corr/rollout_log_ppl:np.float64(0.2411523106857203) - rollout_corr/log_ppl_diff:np.float64(0.0013615062925964594) - rollout_corr/log_ppl_abs_diff:np.float64(0.002768128179013729) - rollout_corr/log_ppl_diff_max:np.float64(0.011708617210388184) - rollout_corr/log_ppl_diff_min:np.float64(-0.010206863284111023) - rollout_corr/ppl_ratio:np.float64(1.0013677747920156) - rollout_corr/chi2_token:np.float64(0.0024920080322772264) - rollout_corr/chi2_seq:np.float64(0.9803334202151746) - actor/pg_loss:np.float64(0.0052698974031955) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015525902631452482) - actor/ppo_kl:np.float64(0.00019933821041862032) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(172811.0304318224) - self_distillation/token_reweight_w_std:np.float64(2611723.788451826) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009630725253373) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09219890262465924) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.5317341759800911) - perf/mfu/actor:np.float64(0.03233182941898891) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.27649307250977) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011396203190088272 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011396203190088272 - critic/returns/max:0.875 - critic/returns/min:-0.87
(TaskRunner pid=2026458) Training Progress: 84%|████████▍ | 84/100 [59:18<09:42, 36.38s/it]
(TaskRunner pid=2026458) step:85 - global_seqlen/min:16721 - global_seqlen/max:24217 - global_seqlen/minmax_diff:7496 - global_seqlen/balanced_min:19463 - global_seqlen/balanced_max:19465 - global_seqlen/mean:19464.25 - actor/entropy:0.23170782625675201 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5486323833465576 - training/rollout_probs_diff_mean:0.005258036311715841 - training/rollout_probs_diff_std:0.013953270390629768 - training/rollout_actor_probs_pearson_corr:0.998067319393158 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0003063678741455 - rollout_corr/rollout_is_ratio_fraction_high:2.5328639821964316e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.065727964392863e-05 - rollout_corr/rollout_is_max:2.629549741744995 - rollout_corr/rollout_is_min:0.32894784212112427 - rollout_corr/rollout_is_std:0.04162293300032616 - rollout_corr/rollout_is_eff_sample_size:0.9982714780308726 - rollout_corr/rollout_is_seq_mean:1.0004109144210815 - rollout_corr/rollout_is_seq_std:0.002749430714175105 - rollout_corr/rollout_is_seq_max:1.0109564065933228 - rollout_corr/rollout_is_seq_min:0.9916076064109802 - rollout_corr/rollout_is_seq_max_deviation:0.010956406593322754 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2771666692569852) - rollout_corr/training_log_ppl:np.float64(0.24135592477978207) - rollout_corr/kl:np.float64(0.0007578132510701607) - rollout_corr/k3_kl:np.float64(0.0010257670675173358) - rollout_corr/rollout_ppl:np.float64(1.2761707119643688) - rollout_corr/rollout_log_ppl:np.float64(0.24059811038023327) - rollout_corr/log_ppl_diff:np.float64(0.0007578143995488063) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024299477954627946) - rollout_corr/log_ppl_diff_max:np.float64(0.011833712458610535) - rollout_corr/log_ppl_diff_min:np.float64(-0.010344773530960083) - rollout_corr/ppl_ratio:np.float64(1.0007629161700606) - rollout_corr/chi2_token:np.float64(0.0026269492227584124) - rollout_corr/chi2_seq:np.float64(1.3403879939578474) - actor/pg_loss:np.float64(0.002229055855423212) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019035061723116087) - actor/ppo_kl:np.float64(0.00024953764273760726) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(17317.446490210947) - self_distillation/token_reweight_w_std:np.float64(276465.0479241874) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991332795470953) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03143878703122027) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2347664088010788) - perf/mfu/actor:np.float64(0.03632172645099713) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.27149963378906) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004859932523686439 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004859932523686439 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:308.4453
(TaskRunner pid=2026458) Training Progress: 85%|████████▌ | 85/100 [59:48<08:38, 34.60s/it]
(TaskRunner pid=2026458) step:86 - global_seqlen/min:16265 - global_seqlen/max:20200 - global_seqlen/minmax_diff:3935 - global_seqlen/balanced_min:18647 - global_seqlen/balanced_max:18651 - global_seqlen/mean:18649.75 - actor/entropy:0.23041389882564545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.898743212223053 - training/rollout_probs_diff_mean:0.005259583238512278 - training/rollout_probs_diff_std:0.014470859430730343 - training/rollout_actor_probs_pearson_corr:0.9979584813117981 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.9998511075973511 - rollout_corr/rollout_is_ratio_fraction_high:3.836807809420861e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.394679803634062e-05 - rollout_corr/rollout_is_max:2.1972851753234863 - rollout_corr/rollout_is_min:0.0777815580368042 - rollout_corr/rollout_is_std:0.04242711886763573 - rollout_corr/rollout_is_eff_sample_size:0.9982026989977762 - rollout_corr/rollout_is_seq_mean:0.9998696446418762 - rollout_corr/rollout_is_seq_std:0.0025317175313830376 - rollout_corr/rollout_is_seq_max:1.009598731994629 - rollout_corr/rollout_is_seq_min:0.9924889802932739 - rollout_corr/rollout_is_seq_max_deviation:0.009598731994628906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2799476818181574) - rollout_corr/training_log_ppl:np.float64(0.24274212954333052) - rollout_corr/kl:np.float64(0.0012324780372026112) - rollout_corr/k3_kl:np.float64(0.0011905511769896293) - rollout_corr/rollout_ppl:np.float64(1.2783302562311292) - rollout_corr/rollout_log_ppl:np.float64(0.24150965179433115) - rollout_corr/log_ppl_diff:np.float64(0.0012324777489993721) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024851207563187927) - rollout_corr/log_ppl_diff_max:np.float64(0.009878866374492645) - rollout_corr/log_ppl_diff_min:np.float64(-0.00684240460395813) - rollout_corr/ppl_ratio:np.float64(1.0012374119833112) - rollout_corr/chi2_token:np.float64(0.0022098186891525984) - rollout_corr/chi2_seq:np.float64(0.6068784659728408) - actor/pg_loss:np.float64(0.0033142786705866456) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009858607454589219) - actor/ppo_kl:np.float64(0.0001475137922213321) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(165894.66037644562) - self_distillation/token_reweight_w_std:np.float64(2346342.531521928) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011869312729686) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07381428830558434) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.42134569585323334) - perf/mfu/actor:np.float64(0.0348608857845462) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1626739501953) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0060413735918700695 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0060413735918700695 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:305.
(TaskRunner pid=2026458) Training Progress: 86%|████████▌ | 86/100 [1:00:19<07:46, 33.35s/it]
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) step:87 - global_seqlen/min:16648 - global_seqlen/max:20097 - global_seqlen/minmax_diff:3449 - global_seqlen/balanced_min:18341 - global_seqlen/balanced_max:18344 - global_seqlen/mean:18342.375 - actor/entropy:0.24005606770515442 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44461295008659363 - training/rollout_probs_diff_mean:0.005454737227410078 - training/rollout_probs_diff_std:0.014133700169622898 - training/rollout_actor_probs_pearson_corr:0.998093843460083 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.53125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.9998968243598938 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.625958146993071e-05 - rollout_corr/rollout_is_max:1.91322922706604 - rollout_corr/rollout_is_min:0.3312951922416687 - rollout_corr/rollout_is_std:0.0426471121609211 - rollout_corr/rollout_is_eff_sample_size:0.998184050703313 - rollout_corr/rollout_is_seq_mean:0.999923050403595 - rollout_corr/rollout_is_seq_std:0.0029033836908638477 - rollout_corr/rollout_is_seq_max:1.0088640451431274 - rollout_corr/rollout_is_seq_min:0.990547776222229 - rollout_corr/rollout_is_seq_max_deviation:0.009452223777770996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2908267816528678) - rollout_corr/training_log_ppl:np.float64(0.25219142163405195) - rollout_corr/kl:np.float64(0.0013515868227500505) - rollout_corr/k3_kl:np.float64(0.001383622248795291) - rollout_corr/rollout_ppl:np.float64(1.2890665112063289) - rollout_corr/rollout_log_ppl:np.float64(0.25083983279182576) - rollout_corr/log_ppl_diff:np.float64(0.0013515888422261924) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028635623457375914) - rollout_corr/log_ppl_diff_max:np.float64(0.012585937976837158) - rollout_corr/log_ppl_diff_min:np.float64(-0.009711101651191711) - rollout_corr/ppl_ratio:np.float64(1.0013586606364697) - rollout_corr/chi2_token:np.float64(0.0029732428956776857) - rollout_corr/chi2_seq:np.float64(0.6213931296952069) - actor/pg_loss:np.float64(0.005519538884982467) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011970178129558917) - actor/ppo_kl:np.float64(0.00032003364054844496) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53125) - self_distillation/token_reweight_w_mean:np.float64(104019.92419685167) - self_distillation/token_reweight_w_std:np.float64(1491533.3107083726) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000025857705623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07910635319421999) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.44405730813741684) - perf/mfu/actor:np.float64(0.03427082030528879) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2020378112793) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009922783821821213 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009922783821821213 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:277.73046875
(TaskRunner pid=2026458) Training Progress: 87%|████████▋ | 87/100 [1:00:48<06:58, 32.18s/it]
(TaskRunner pid=2026458) step:88 - global_seqlen/min:14981 - global_seqlen/max:20248 - global_seqlen/minmax_diff:5267 - global_seqlen/balanced_min:17171 - global_seqlen/balanced_max:17176 - global_seqlen/mean:17174.125 - actor/entropy:0.21586236357688904 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5279986262321472 - training/rollout_probs_diff_mean:0.005149980541318655 - training/rollout_probs_diff_std:0.014039847999811172 - training/rollout_actor_probs_pearson_corr:0.9979734420776367 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9998298287391663 - rollout_corr/rollout_is_ratio_fraction_high:2.9164297302486375e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4582148651243187e-05 - rollout_corr/rollout_is_max:2.4087512493133545 - rollout_corr/rollout_is_min:0.3157358169555664 - rollout_corr/rollout_is_std:0.04135211929678917 - rollout_corr/rollout_is_eff_sample_size:0.998292149229813 - rollout_corr/rollout_is_seq_mean:0.9997440576553345 - rollout_corr/rollout_is_seq_std:0.0026299559976905584 - rollout_corr/rollout_is_seq_max:1.0093685388565063 - rollout_corr/rollout_is_seq_min:0.9928037524223328 - rollout_corr/rollout_is_seq_max_deviation:0.009368538856506348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.263124991208315) - rollout_corr/training_log_ppl:np.float64(0.23035154843819328) - rollout_corr/kl:np.float64(0.0012507079140435806) - rollout_corr/k3_kl:np.float64(0.0012822421521718752) - rollout_corr/rollout_ppl:np.float64(1.2614940297789872) - rollout_corr/rollout_log_ppl:np.float64(0.22910083959868643) - rollout_corr/log_ppl_diff:np.float64(0.0012507088395068422) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027966901106992736) - rollout_corr/log_ppl_diff_max:np.float64(0.014727145433425903) - rollout_corr/log_ppl_diff_min:np.float64(-0.014272093772888184) - rollout_corr/ppl_ratio:np.float64(1.0012579679023474) - rollout_corr/chi2_token:np.float64(0.002700346987694502) - rollout_corr/chi2_seq:np.float64(0.8957935450598598) - actor/pg_loss:np.float64(0.0045979529386386275) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008685116490596556) - actor/ppo_kl:np.float64(0.00010619230280539682) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(126031.5825594815) - self_distillation/token_reweight_w_std:np.float64(1914441.9060592402) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010992304887623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07210871722782031) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4531179219484329) - perf/mfu/actor:np.float64(0.032191482757026275) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.15955352783203) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014600376598536968 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014600376598536968 - critic/returns/max:0.875 - critic/returns/min:-0.875
(TaskRunner pid=2026458) Training Progress: 88%|████████▊ | 88/100 [1:01:18<06:17, 31.49s/it]
(TaskRunner pid=2026458) step:89 - global_seqlen/min:16152 - global_seqlen/max:22412 - global_seqlen/minmax_diff:6260 - global_seqlen/balanced_min:18887 - global_seqlen/balanced_max:18895 - global_seqlen/mean:18893.375 - actor/entropy:0.22386007010936737 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2151239514350891 - training/rollout_probs_diff_mean:0.005173899233341217 - training/rollout_probs_diff_std:0.013607512228190899 - training/rollout_actor_probs_pearson_corr:0.9981110692024231 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000429153442383 - rollout_corr/rollout_is_ratio_fraction_high:2.6122277631657198e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.83668365329504e-05 - rollout_corr/rollout_is_max:2.51365327835083 - rollout_corr/rollout_is_min:0.1054598018527031 - rollout_corr/rollout_is_std:0.04179442301392555 - rollout_corr/rollout_is_eff_sample_size:0.9982565097660324 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0027215098962187767 - rollout_corr/rollout_is_seq_max:1.0083972215652466 - rollout_corr/rollout_is_seq_min:0.9915516376495361 - rollout_corr/rollout_is_seq_max_deviation:0.008448362350463867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2823212435469031) - rollout_corr/training_log_ppl:np.float64(0.24520724279864226) - rollout_corr/kl:np.float64(0.0008697826999508607) - rollout_corr/k3_kl:np.float64(0.0010809666277964425) - rollout_corr/rollout_ppl:np.float64(1.2811899655498564) - rollout_corr/rollout_log_ppl:np.float64(0.24433745855640154) - rollout_corr/log_ppl_diff:np.float64(0.000869784242240712) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022608327853959054) - rollout_corr/log_ppl_diff_max:np.float64(0.010239839553833008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071149468421936035) - rollout_corr/ppl_ratio:np.float64(1.0008741298224777) - rollout_corr/chi2_token:np.float64(0.0025997732300311327) - rollout_corr/chi2_seq:np.float64(0.5740627928171307) - actor/pg_loss:np.float64(0.0036603367188945413) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008643267137813382) - actor/ppo_kl:np.float64(-9.36468737013918e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(95742.81214469112) - self_distillation/token_reweight_w_std:np.float64(1397763.4353546314) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014505761209875) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07612263548071496) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.43359871953725815) - perf/mfu/actor:np.float64(0.035258064824575705) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1541633605957) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00761464424431324 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00761464424431324 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/
(TaskRunner pid=2026458) Training Progress: 89%|████████▉ | 89/100 [1:01:50<05:46, 31.47s/it]
(TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90}
(TaskRunner pid=2026458) validation generation end
(TaskRunner pid=2026458) [prompt] system
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2026458) <reasoning>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) user
(TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2026458) Please reason step by step.
(TaskRunner pid=2026458) assistant
(TaskRunner pid=2026458) <think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) </think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) [response] <reasoning>
(TaskRunner pid=2026458) The question asks which of the given options is the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a chlorine atom, a trifluoromethyl group (C(F)(F)F), and a specific aromatic ring structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option A contains a structure with a chlorine atom, a trifluoromethyl group, and the correct aromatic ring structure. The formula is "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", which matches the required components of the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option B contains a chlorine atom but does not have the correct trifluoromethyl group or the correct aromatic ring structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option C contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option D contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it contains all the necessary components of the product.
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458) [ground_truth] A
(TaskRunner pid=2026458) [score] 1.0
(TaskRunner pid=2026458) [acc] 1.0
(TaskRunner pid=2026458) [pred] A
(TaskRunner pid=2026458) [incorrect_format] 1
(TaskRunner pid=2026458) [feedback]
(TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90
(WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor
(TaskRunner pid=2026458) step:90 - global_seqlen/min:14894 - global_seqlen/max:19933 - global_seqlen/minmax_diff:5039 - global_seqlen/balanced_min:18191 - global_seqlen/balanced_max:18194 - global_seqlen/mean:18192.75 - actor/entropy:0.21710193157196045 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3145221471786499 - training/rollout_probs_diff_mean:0.004972291644662619 - training/rollout_probs_diff_std:0.013361528515815735 - training/rollout_actor_probs_pearson_corr:0.9982044100761414 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:1.0000026226043701 - rollout_corr/rollout_is_ratio_fraction_high:2.6879552024183795e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0319329855265096e-05 - rollout_corr/rollout_is_max:3.3088538646698 - rollout_corr/rollout_is_min:0.46167826652526855 - rollout_corr/rollout_is_std:0.041136063635349274 - rollout_corr/rollout_is_eff_sample_size:0.9983105639498434 - rollout_corr/rollout_is_seq_mean:1.0000332593917847 - rollout_corr/rollout_is_seq_std:0.0027625819202512503 - rollout_corr/rollout_is_seq_max:1.0086408853530884 - rollout_corr/rollout_is_seq_min:0.9904849529266357 - rollout_corr/rollout_is_seq_max_deviation:0.009515047073364258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2710709455423057) - rollout_corr/training_log_ppl:np.float64(0.23564479569904506) - rollout_corr/kl:np.float64(0.0014168473381213076) - rollout_corr/k3_kl:np.float64(0.0014046167735841664) - rollout_corr/rollout_ppl:np.float64(1.2692075530067086) - rollout_corr/rollout_log_ppl:np.float64(0.2342279483564198) - rollout_corr/log_ppl_diff:np.float64(0.0014168473426252604) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028690413455478847) - rollout_corr/log_ppl_diff_max:np.float64(0.03278002142906189) - rollout_corr/log_ppl_diff_min:np.float64(-0.008600443601608276) - rollout_corr/ppl_ratio:np.float64(1.0014270655810833) - rollout_corr/chi2_token:np.float64(0.002751560416072607) - rollout_corr/chi2_seq:np.float64(0.7767476551234722) - actor/pg_loss:np.float64(0.0019601467065513134) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009453206475882325) - actor/ppo_kl:np.float64(0.0005355499455532708) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(99385.55734024942) - self_distillation/token_reweight_w_std:np.float64(1449312.0253919773) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012841576244682) - self_distillation/token_reweight_w_clip_frac:np.float64(0.045599023724207655) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.30510950461030006) - perf/mfu/actor:np.float64(0.03390671346645702) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.11214065551758) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/reward/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.11
(TaskRunner pid=2026458) Training Progress: 90%|█████████ | 90/100 [1:04:17<11:03, 66.39s/it]
(TaskRunner pid=2026458) step:91 - global_seqlen/min:15865 - global_seqlen/max:25244 - global_seqlen/minmax_diff:9379 - global_seqlen/balanced_min:18811 - global_seqlen/balanced_max:18818 - global_seqlen/mean:18815.125 - actor/entropy:0.2314702272415161 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21055352687835693 - training/rollout_probs_diff_mean:0.005051769316196442 - training/rollout_probs_diff_std:0.013311025686562061 - training/rollout_actor_probs_pearson_corr:0.9982390999794006 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999205470085144 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.47056513116695e-05 - rollout_corr/rollout_is_max:1.7587093114852905 - rollout_corr/rollout_is_min:0.23516227304935455 - rollout_corr/rollout_is_std:0.039755210280418396 - rollout_corr/rollout_is_eff_sample_size:0.9984218983709217 - rollout_corr/rollout_is_seq_mean:0.9999246001243591 - rollout_corr/rollout_is_seq_std:0.002499330323189497 - rollout_corr/rollout_is_seq_max:1.0098474025726318 - rollout_corr/rollout_is_seq_min:0.9917200207710266 - rollout_corr/rollout_is_seq_max_deviation:0.009847402572631836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2789378436282277) - rollout_corr/training_log_ppl:np.float64(0.24201013351557776) - rollout_corr/kl:np.float64(0.0009981181076650358) - rollout_corr/k3_kl:np.float64(0.0010161043154539584) - rollout_corr/rollout_ppl:np.float64(1.2776579768396914) - rollout_corr/rollout_log_ppl:np.float64(0.2410120142158121) - rollout_corr/log_ppl_diff:np.float64(0.0009981192997656763) - rollout_corr/log_ppl_abs_diff:np.float64(0.002134583191946149) - rollout_corr/log_ppl_diff_max:np.float64(0.00839272141456604) - rollout_corr/log_ppl_diff_min:np.float64(-0.010266929864883423) - rollout_corr/ppl_ratio:np.float64(1.0010020262561738) - rollout_corr/chi2_token:np.float64(0.0021203176584094763) - rollout_corr/chi2_seq:np.float64(0.651163290720433) - actor/pg_loss:np.float64(0.002993874717503786) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000568103197565506) - actor/ppo_kl:np.float64(6.333214638765128e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(82354.48971123807) - self_distillation/token_reweight_w_std:np.float64(1252927.9291606762) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009026953484863) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06239277429995127) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.33179403841495514) - perf/mfu/actor:np.float64(0.034982824171721626) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.52531814575195) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002352050505578518 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002352050505578518 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_leng
(TaskRunner pid=2026458) Training Progress: 91%|█████████ | 91/100 [1:04:47<08:18, 55.36s/it]
(TaskRunner pid=2026458) step:92 - global_seqlen/min:16281 - global_seqlen/max:21475 - global_seqlen/minmax_diff:5194 - global_seqlen/balanced_min:18650 - global_seqlen/balanced_max:18651 - global_seqlen/mean:18650.625 - actor/entropy:0.22619383037090302 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.005189774092286825 - training/rollout_probs_diff_std:0.013963738456368446 - training/rollout_actor_probs_pearson_corr:0.9980257153511047 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999138116836548 - rollout_corr/rollout_is_ratio_fraction_high:1.3226638657215517e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011903974518645555 - rollout_corr/rollout_is_max:2.0864930152893066 - rollout_corr/rollout_is_min:0.0921701192855835 - rollout_corr/rollout_is_std:0.04230894520878792 - rollout_corr/rollout_is_eff_sample_size:0.9982129142824776 - rollout_corr/rollout_is_seq_mean:0.9999446272850037 - rollout_corr/rollout_is_seq_std:0.0027264179661870003 - rollout_corr/rollout_is_seq_max:1.0081113576889038 - rollout_corr/rollout_is_seq_min:0.9917219281196594 - rollout_corr/rollout_is_seq_max_deviation:0.008278071880340576 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2724861456081271) - rollout_corr/training_log_ppl:np.float64(0.2372018521418795) - rollout_corr/kl:np.float64(0.0009739822945320498) - rollout_corr/k3_kl:np.float64(0.0010123744499423992) - rollout_corr/rollout_ppl:np.float64(1.2711997576989233) - rollout_corr/rollout_log_ppl:np.float64(0.23622786854684819) - rollout_corr/log_ppl_diff:np.float64(0.0009739835950313136) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024230604904005304) - rollout_corr/log_ppl_diff_max:np.float64(0.012229472398757935) - rollout_corr/log_ppl_diff_min:np.float64(-0.010651081800460815) - rollout_corr/ppl_ratio:np.float64(1.0009791306219995) - rollout_corr/chi2_token:np.float64(0.0021006297320127487) - rollout_corr/chi2_seq:np.float64(0.7930810896214098) - actor/pg_loss:np.float64(0.002591841504909098) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004880208070972003) - actor/ppo_kl:np.float64(-2.9975729340492308e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(72610.33176031034) - self_distillation/token_reweight_w_std:np.float64(1092734.639293266) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988934537395835) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06326231267303228) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.3366621397435665) - perf/mfu/actor:np.float64(0.03433711138530011) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.60525131225586) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006365319713950157 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006365319713950157 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_leng
(TaskRunner pid=2026458) Training Progress: 92%|█████████▏| 92/100 [1:05:17<06:22, 47.76s/it]
(TaskRunner pid=2026458) step:93 - global_seqlen/min:15058 - global_seqlen/max:22608 - global_seqlen/minmax_diff:7550 - global_seqlen/balanced_min:18025 - global_seqlen/balanced_max:18028 - global_seqlen/mean:18026.625 - actor/entropy:0.20888566970825195 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24053943157196045 - training/rollout_probs_diff_mean:0.004918403923511505 - training/rollout_probs_diff_std:0.013119952753186226 - training/rollout_actor_probs_pearson_corr:0.9981909394264221 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.98828125 - self_distillation/correct_sample_fraction:0.87890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.98828125 - rollout_corr/rollout_is_mean:0.9998151659965515 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.263365528662689e-05 - rollout_corr/rollout_is_max:1.6370371580123901 - rollout_corr/rollout_is_min:0.2820625901222229 - rollout_corr/rollout_is_std:0.03976345434784889 - rollout_corr/rollout_is_eff_sample_size:0.9984207100389592 - rollout_corr/rollout_is_seq_mean:0.9998314380645752 - rollout_corr/rollout_is_seq_std:0.0023552256170660257 - rollout_corr/rollout_is_seq_max:1.0079485177993774 - rollout_corr/rollout_is_seq_min:0.9893273115158081 - rollout_corr/rollout_is_seq_max_deviation:0.010672688484191895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2420122399926186) - rollout_corr/training_log_ppl:np.float64(0.2130839267920237) - rollout_corr/kl:np.float64(0.0009293865148691793) - rollout_corr/k3_kl:np.float64(0.000889873810649533) - rollout_corr/rollout_ppl:np.float64(1.2408293401822448) - rollout_corr/rollout_log_ppl:np.float64(0.2121545401896583) - rollout_corr/log_ppl_diff:np.float64(0.0009293866023654118) - rollout_corr/log_ppl_abs_diff:np.float64(0.00208455762185622) - rollout_corr/log_ppl_diff_max:np.float64(0.008339852094650269) - rollout_corr/log_ppl_diff_min:np.float64(-0.005493566393852234) - rollout_corr/ppl_ratio:np.float64(1.0009328972082585) - rollout_corr/chi2_token:np.float64(0.00168294133618474) - rollout_corr/chi2_seq:np.float64(0.3678914944175631) - actor/pg_loss:np.float64(0.00139771425165236) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002747084317888948) - actor/ppo_kl:np.float64(-7.94472919167788e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.98828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.98828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.87890625) - self_distillation/token_reweight_w_mean:np.float64(72035.54537807149) - self_distillation/token_reweight_w_std:np.float64(1075218.813784783) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001510763540864) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03214944276260212) - self_distillation/empty_target_batch:np.float64(0.01171875) - actor/grad_norm:np.float64(0.2579457201063633) - perf/mfu/actor:np.float64(0.03364477703539987) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.39699172973633) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.87890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.87890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036366567946970463 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.0036366567946970463 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:29
(TaskRunner pid=2026458) Training Progress: 93%|█████████▎| 93/100 [1:05:47<04:56, 42.29s/it]
(TaskRunner pid=2026458) step:94 - global_seqlen/min:15786 - global_seqlen/max:20010 - global_seqlen/minmax_diff:4224 - global_seqlen/balanced_min:17735 - global_seqlen/balanced_max:17737 - global_seqlen/mean:17736.0 - actor/entropy:0.22735926508903503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311311483383179 - training/rollout_probs_diff_mean:0.005040409974753857 - training/rollout_probs_diff_std:0.013486634008586407 - training/rollout_actor_probs_pearson_corr:0.9981728792190552 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:0.9998102784156799 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0001036699177348055 - rollout_corr/rollout_is_max:1.7523070573806763 - rollout_corr/rollout_is_min:0.05793754756450653 - rollout_corr/rollout_is_std:0.03992203250527382 - rollout_corr/rollout_is_eff_sample_size:0.9984081138940653 - rollout_corr/rollout_is_seq_mean:0.9999222159385681 - rollout_corr/rollout_is_seq_std:0.002507504541426897 - rollout_corr/rollout_is_seq_max:1.0082566738128662 - rollout_corr/rollout_is_seq_min:0.9931005835533142 - rollout_corr/rollout_is_seq_max_deviation:0.008256673812866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2725239442661405) - rollout_corr/training_log_ppl:np.float64(0.23770636528206524) - rollout_corr/kl:np.float64(0.001132996877289294) - rollout_corr/k3_kl:np.float64(0.0012387144401486694) - rollout_corr/rollout_ppl:np.float64(1.2710481900721788) - rollout_corr/rollout_log_ppl:np.float64(0.23657336641917937) - rollout_corr/log_ppl_diff:np.float64(0.00113299886288587) - rollout_corr/log_ppl_abs_diff:np.float64(0.002540186673286371) - rollout_corr/log_ppl_diff_max:np.float64(0.02026122808456421) - rollout_corr/log_ppl_diff_min:np.float64(-0.015430659055709839) - rollout_corr/ppl_ratio:np.float64(1.0011393770109862) - rollout_corr/chi2_token:np.float64(0.002516603097319603) - rollout_corr/chi2_seq:np.float64(0.9786048375535756) - actor/pg_loss:np.float64(0.002404728322289884) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001062956328951259) - actor/ppo_kl:np.float64(0.0001939034154609942) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(96479.46470297547) - self_distillation/token_reweight_w_std:np.float64(1265708.629157153) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990413470659405) - self_distillation/token_reweight_w_clip_frac:np.float64(0.054301650350680575) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.35001390427351) - perf/mfu/actor:np.float64(0.032954674991088666) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.4532012939453) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00508630508556962 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00508630508556962 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:301.4375 - response
(TaskRunner pid=2026458) Training Progress: 94%|█████████▍| 94/100 [1:06:16<03:50, 38.44s/it]
(TaskRunner pid=2026458) step:95 - global_seqlen/min:14931 - global_seqlen/max:21336 - global_seqlen/minmax_diff:6405 - global_seqlen/balanced_min:17803 - global_seqlen/balanced_max:17812 - global_seqlen/mean:17810.75 - actor/entropy:0.22719933092594147 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3546398878097534 - training/rollout_probs_diff_mean:0.004889350384473801 - training/rollout_probs_diff_std:0.012968857772648335 - training/rollout_actor_probs_pearson_corr:0.9983217716217041 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:0.9999182224273682 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3313806448422838e-05 - rollout_corr/rollout_is_max:1.646693468093872 - rollout_corr/rollout_is_min:0.012675890699028969 - rollout_corr/rollout_is_std:0.03914937749505043 - rollout_corr/rollout_is_eff_sample_size:0.9984694339690914 - rollout_corr/rollout_is_seq_mean:0.9998860955238342 - rollout_corr/rollout_is_seq_std:0.002519140485674143 - rollout_corr/rollout_is_seq_max:1.0078048706054688 - rollout_corr/rollout_is_seq_min:0.993129551410675 - rollout_corr/rollout_is_seq_max_deviation:0.00780487060546875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.269720992539078) - rollout_corr/training_log_ppl:np.float64(0.23537427955307066) - rollout_corr/kl:np.float64(0.000860856562256096) - rollout_corr/k3_kl:np.float64(0.0010757347620256041) - rollout_corr/rollout_ppl:np.float64(1.2686140895821154) - rollout_corr/rollout_log_ppl:np.float64(0.2345134206698276) - rollout_corr/log_ppl_diff:np.float64(0.0008608588832430542) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024572820402681828) - rollout_corr/log_ppl_diff_max:np.float64(0.022655338048934937) - rollout_corr/log_ppl_diff_min:np.float64(-0.008849263191223145) - rollout_corr/ppl_ratio:np.float64(1.0008665048517287) - rollout_corr/chi2_token:np.float64(0.002582071116194129) - rollout_corr/chi2_seq:np.float64(1.1929384064860642) - actor/pg_loss:np.float64(0.0030269279377534986) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001186676310680923) - actor/ppo_kl:np.float64(-0.00011882417348374474) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(142327.77909193072) - self_distillation/token_reweight_w_std:np.float64(1811469.8086368854) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001867031678557) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07137267180951312) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.4086311012506485) - perf/mfu/actor:np.float64(0.03299815181672398) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.32865142822266) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007821861072443426 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007821861072443426 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.39
(TaskRunner pid=2026458) Training Progress: 95%|█████████▌| 95/100 [1:06:46<02:59, 35.91s/it]
(TaskRunner pid=2026458) step:96 - global_seqlen/min:15504 - global_seqlen/max:21947 - global_seqlen/minmax_diff:6443 - global_seqlen/balanced_min:17847 - global_seqlen/balanced_max:17850 - global_seqlen/mean:17848.25 - actor/entropy:0.24302901327610016 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7704145312309265 - training/rollout_probs_diff_mean:0.0050596678629517555 - training/rollout_probs_diff_std:0.013549817726016045 - training/rollout_actor_probs_pearson_corr:0.9982749223709106 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000941753387451 - rollout_corr/rollout_is_ratio_fraction_high:4.025656744488515e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.709428271278739e-05 - rollout_corr/rollout_is_max:2.5671136379241943 - rollout_corr/rollout_is_min:0.13419120013713837 - rollout_corr/rollout_is_std:0.039693690836429596 - rollout_corr/rollout_is_eff_sample_size:0.9984272458997584 - rollout_corr/rollout_is_seq_mean:1.000090479850769 - rollout_corr/rollout_is_seq_std:0.0026221859734505415 - rollout_corr/rollout_is_seq_max:1.009491205215454 - rollout_corr/rollout_is_seq_min:0.9908090233802795 - rollout_corr/rollout_is_seq_max_deviation:0.009491205215454102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2904391190968454) - rollout_corr/training_log_ppl:np.float64(0.2496467249293346) - rollout_corr/kl:np.float64(0.0008864650635374005) - rollout_corr/k3_kl:np.float64(0.0009545528546368587) - rollout_corr/rollout_ppl:np.float64(1.2892436645925045) - rollout_corr/rollout_log_ppl:np.float64(0.24876026091806125) - rollout_corr/log_ppl_diff:np.float64(0.0008864640112733468) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021889270428800955) - rollout_corr/log_ppl_diff_max:np.float64(0.013020426034927368) - rollout_corr/log_ppl_diff_min:np.float64(-0.006367728114128113) - rollout_corr/ppl_ratio:np.float64(1.0008906812872738) - rollout_corr/chi2_token:np.float64(0.0020458281505852938) - rollout_corr/chi2_seq:np.float64(0.6757269599474967) - actor/pg_loss:np.float64(0.0028436718275770545) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006703252311126562) - actor/ppo_kl:np.float64(0.0001225946314420412) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(73408.10212582699) - self_distillation/token_reweight_w_std:np.float64(1221313.2646529041) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004472548607737) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07176223069836851) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.3494473174214363) - perf/mfu/actor:np.float64(0.03336222886368918) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.36438751220703) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0025512599386274815 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0025512599386274815 - critic/returns/max:0.875 - critic/returns/min:-0.87
(TaskRunner pid=2026458) Training Progress: 96%|█████████▌| 96/100 [1:07:15<02:15, 33.85s/it]
(TaskRunner pid=2026458) step:97 - global_seqlen/min:16071 - global_seqlen/max:23461 - global_seqlen/minmax_diff:7390 - global_seqlen/balanced_min:18811 - global_seqlen/balanced_max:18815 - global_seqlen/mean:18813.875 - actor/entropy:0.2445484697818756 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4984980821609497 - training/rollout_probs_diff_mean:0.005207015201449394 - training/rollout_probs_diff_std:0.013489038683474064 - training/rollout_actor_probs_pearson_corr:0.9982696175575256 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.52734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:0.9999282360076904 - rollout_corr/rollout_is_ratio_fraction_high:1.330795930698514e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.65397965349257e-05 - rollout_corr/rollout_is_max:3.3646888732910156 - rollout_corr/rollout_is_min:0.3885670304298401 - rollout_corr/rollout_is_std:0.04048614203929901 - rollout_corr/rollout_is_eff_sample_size:0.9983633169728505 - rollout_corr/rollout_is_seq_mean:0.9999846816062927 - rollout_corr/rollout_is_seq_std:0.002680700272321701 - rollout_corr/rollout_is_seq_max:1.0078043937683105 - rollout_corr/rollout_is_seq_min:0.9888970255851746 - rollout_corr/rollout_is_seq_max_deviation:0.01110297441482544 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3025756301358342) - rollout_corr/training_log_ppl:np.float64(0.2600360629003262) - rollout_corr/kl:np.float64(0.001345034299074399) - rollout_corr/k3_kl:np.float64(0.0010471967440253138) - rollout_corr/rollout_ppl:np.float64(1.3007347490638494) - rollout_corr/rollout_log_ppl:np.float64(0.2586910269019427) - rollout_corr/log_ppl_diff:np.float64(0.001345035998383537) - rollout_corr/log_ppl_abs_diff:np.float64(0.002600655221613124) - rollout_corr/log_ppl_diff_max:np.float64(0.013873368501663208) - rollout_corr/log_ppl_diff_min:np.float64(-0.008637338876724243) - rollout_corr/ppl_ratio:np.float64(1.0013508247211576) - rollout_corr/chi2_token:np.float64(0.001424239482730627) - rollout_corr/chi2_seq:np.float64(0.4958528857678175) - actor/pg_loss:np.float64(0.0023965955479070544) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000528412345374818) - actor/ppo_kl:np.float64(0.00046769405014401855) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.52734375) - self_distillation/token_reweight_w_mean:np.float64(76633.93915633648) - self_distillation/token_reweight_w_std:np.float64(1211350.6270655608) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995253349188715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0652933964447584) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.31651656329631805) - perf/mfu/actor:np.float64(0.03500956607334557) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.4477882385254) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004978840239346027 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004978840239346027 - critic/returns/max:0.875 - critic/returns/min:-0.875 - respons
(TaskRunner pid=2026458) Training Progress: 97%|█████████▋| 97/100 [1:07:45<01:37, 32.62s/it]
(TaskRunner pid=2026458) step:98 - global_seqlen/min:14853 - global_seqlen/max:23186 - global_seqlen/minmax_diff:8333 - global_seqlen/balanced_min:19016 - global_seqlen/balanced_max:19021 - global_seqlen/mean:19019.875 - actor/entropy:0.23993536829948425 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2247922420501709 - training/rollout_probs_diff_mean:0.005032053682953119 - training/rollout_probs_diff_std:0.013124685734510422 - training/rollout_actor_probs_pearson_corr:0.99836266040802 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9997885227203369 - rollout_corr/rollout_is_ratio_fraction_high:2.6424617317388766e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.9636925066588446e-05 - rollout_corr/rollout_is_max:2.206674814224243 - rollout_corr/rollout_is_min:0.19827817380428314 - rollout_corr/rollout_is_std:0.04013793542981148 - rollout_corr/rollout_is_eff_sample_size:0.9983908838332385 - rollout_corr/rollout_is_seq_mean:0.9997617602348328 - rollout_corr/rollout_is_seq_std:0.0025118952617049217 - rollout_corr/rollout_is_seq_max:1.0087740421295166 - rollout_corr/rollout_is_seq_min:0.9924179315567017 - rollout_corr/rollout_is_seq_max_deviation:0.008774042129516602 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2941957069560885) - rollout_corr/training_log_ppl:np.float64(0.254570292920107) - rollout_corr/kl:np.float64(0.0012180306670934726) - rollout_corr/k3_kl:np.float64(0.000970762781207668) - rollout_corr/rollout_ppl:np.float64(1.2925921510905027) - rollout_corr/rollout_log_ppl:np.float64(0.25335226202150807) - rollout_corr/log_ppl_diff:np.float64(0.001218030898598954) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023932511976454407) - rollout_corr/log_ppl_diff_max:np.float64(0.008434802293777466) - rollout_corr/log_ppl_diff_min:np.float64(-0.005523428320884705) - rollout_corr/ppl_ratio:np.float64(1.0012223366647959) - rollout_corr/chi2_token:np.float64(0.0014376682229340076) - rollout_corr/chi2_seq:np.float64(0.6267413701862097) - actor/pg_loss:np.float64(0.003052193787880242) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006503687072836328) - actor/ppo_kl:np.float64(0.000135291372203028) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(107416.14729232923) - self_distillation/token_reweight_w_std:np.float64(1626133.0662139303) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999873915920034) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0583975906483829) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.36374419182538986) - perf/mfu/actor:np.float64(0.03551806757350935) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.40508651733398) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001578870927914977 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001578870927914977 - critic/returns/max:0.875 - critic/returns/min:-0.875
(TaskRunner pid=2026458) Training Progress: 98%|█████████▊| 98/100 [1:08:14<01:03, 31.71s/it]
(TaskRunner pid=2026458) step:99 - global_seqlen/min:16270 - global_seqlen/max:23777 - global_seqlen/minmax_diff:7507 - global_seqlen/balanced_min:18894 - global_seqlen/balanced_max:18898 - global_seqlen/mean:18896.125 - actor/entropy:0.23954525589942932 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5315115451812744 - training/rollout_probs_diff_mean:0.005148096010088921 - training/rollout_probs_diff_std:0.013385861180722713 - training/rollout_actor_probs_pearson_corr:0.9983036518096924 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.00002920627594 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.40447464218596e-05 - rollout_corr/rollout_is_max:1.9394363164901733 - rollout_corr/rollout_is_min:0.2951362729072571 - rollout_corr/rollout_is_std:0.04090065881609917 - rollout_corr/rollout_is_eff_sample_size:0.9983299298399158 - rollout_corr/rollout_is_seq_mean:1.0000901222229004 - rollout_corr/rollout_is_seq_std:0.0026464853435754776 - rollout_corr/rollout_is_seq_max:1.00861656665802 - rollout_corr/rollout_is_seq_min:0.9930309057235718 - rollout_corr/rollout_is_seq_max_deviation:0.00861656665802002 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2940249065868556) - rollout_corr/training_log_ppl:np.float64(0.25375668989727274) - rollout_corr/kl:np.float64(0.0008266884788987738) - rollout_corr/k3_kl:np.float64(0.0009614558624662095) - rollout_corr/rollout_ppl:np.float64(1.2929239594377577) - rollout_corr/rollout_log_ppl:np.float64(0.2529299992165761) - rollout_corr/log_ppl_diff:np.float64(0.000826690680696629) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025123875675490126) - rollout_corr/log_ppl_diff_max:np.float64(0.009278476238250732) - rollout_corr/log_ppl_diff_min:np.float64(-0.007872521877288818) - rollout_corr/ppl_ratio:np.float64(1.000831762328744) - rollout_corr/chi2_token:np.float64(0.002216932363808155) - rollout_corr/chi2_seq:np.float64(0.6580137938726693) - actor/pg_loss:np.float64(0.004063432686962187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007904909580247477) - actor/ppo_kl:np.float64(4.468652047506794e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(131557.28193879593) - self_distillation/token_reweight_w_std:np.float64(2127779.07550747) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999837173614651) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07958601297286805) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.4794868528842926) - perf/mfu/actor:np.float64(0.035181320287522105) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.34246444702148) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003239069599658251 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003239069599658251 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:278.12890
(TaskRunner pid=2026458) Training Progress: 99%|█████████▉| 99/100 [1:08:44<00:31, 31.09s/it]
(TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100}
(TaskRunner pid=2026458) validation generation end
(TaskRunner pid=2026458) [prompt] system
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2026458) <reasoning>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) ...
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) user
(TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ?
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1
(TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1
(TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1
(TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F
(TaskRunner pid=2026458) Please reason step by step.
(TaskRunner pid=2026458) assistant
(TaskRunner pid=2026458) <think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) </think>
(TaskRunner pid=2026458)
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) [response] <reasoning>
(TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a chlorine atom, a trifluoromethyl group (C(F)(F)F), and a specific aromatic ring structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains the correct structure with a chlorine atom, a trifluoromethyl group, and the necessary aromatic ring, matching the product structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option B: "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" contains a chlorine atom but lacks the correct trifluoromethyl group and the correct aromatic ring structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option C: "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" contains a trifluoromethyl group and a chlorine atom but has an incorrect aromatic ring structure and additional functional groups not present in the product.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Option D: "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure.
(TaskRunner pid=2026458)
(TaskRunner pid=2026458) Therefore, the only option that matches the product structure is Option A.
(TaskRunner pid=2026458) </reasoning>
(TaskRunner pid=2026458) <answer>
(TaskRunner pid=2026458) A
(TaskRunner pid=2026458) </answer>
(TaskRunner pid=2026458) [ground_truth] A
(TaskRunner pid=2026458) [score] 1.0
(TaskRunner pid=2026458) [acc] 1.0
(TaskRunner pid=2026458) [pred] A
(TaskRunner pid=2026458) [incorrect_format] 1
(TaskRunner pid=2026458) [feedback]
(TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100
(WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor
(TaskRunner pid=2026458) step:100 - global_seqlen/min:17070 - global_seqlen/max:22010 - global_seqlen/minmax_diff:4940 - global_seqlen/balanced_min:19621 - global_seqlen/balanced_max:19622 - global_seqlen/mean:19621.75 - actor/entropy:0.21838146448135376 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163663625717163 - training/rollout_probs_diff_mean:0.004920980893075466 - training/rollout_probs_diff_std:0.013346283696591854 - training/rollout_actor_probs_pearson_corr:0.9981935024261475 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000070333480835 - rollout_corr/rollout_is_ratio_fraction_high:1.2321644135226961e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.16082179476507e-05 - rollout_corr/rollout_is_max:2.1037662029266357 - rollout_corr/rollout_is_min:0.39213550090789795 - rollout_corr/rollout_is_std:0.04028838127851486 - rollout_corr/rollout_is_eff_sample_size:0.9983795954951445 - rollout_corr/rollout_is_seq_mean:1.0000463724136353 - rollout_corr/rollout_is_seq_std:0.002407139865681529 - rollout_corr/rollout_is_seq_max:1.0090001821517944 - rollout_corr/rollout_is_seq_min:0.9934689998626709 - rollout_corr/rollout_is_seq_max_deviation:0.009000182151794434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2685748748481274) - rollout_corr/training_log_ppl:np.float64(0.23310650100756902) - rollout_corr/kl:np.float64(0.0009287315268036167) - rollout_corr/k3_kl:np.float64(0.001084987000297133) - rollout_corr/rollout_ppl:np.float64(1.2673596912063658) - rollout_corr/rollout_log_ppl:np.float64(0.2321777678298531) - rollout_corr/log_ppl_diff:np.float64(0.0009287331777159125) - rollout_corr/log_ppl_abs_diff:np.float64(0.002280793822137639) - rollout_corr/log_ppl_diff_max:np.float64(0.015468984842300415) - rollout_corr/log_ppl_diff_min:np.float64(-0.010868668556213379) - rollout_corr/ppl_ratio:np.float64(1.0009334608912468) - rollout_corr/chi2_token:np.float64(0.002497109118849039) - rollout_corr/chi2_seq:np.float64(1.0585334873758256) - actor/pg_loss:np.float64(0.0023272527614608407) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004233177678543143) - actor/ppo_kl:np.float64(0.00010989303795483352) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(73094.00544022233) - self_distillation/token_reweight_w_std:np.float64(1188161.9467263469) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000056762015447) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03758277394808829) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.25844827108085155) - perf/mfu/actor:np.float64(0.03632957727570209) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.29161834716797) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/reward/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.10098054289944847)
(TaskRunner pid=2026458) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': "
(TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/reward/std@16': "
(TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/reward/best@2/mean': "
(TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/reward/best@2/std': "
(TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/reward/worst@2/mean': "
(TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/reward/worst@2/std': "
(TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/reward/maj@2/mean': "
(TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/reward/maj@2/std': "
(TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/reward/best@4/mean': "
(TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/reward/best@4/std': "
(TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/reward/worst@4/mean': "
(TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/reward/worst@4/std': "
(TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/reward/maj@4/mean': "
(TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/reward/maj@4/std': "
(TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/reward/best@8/mean': "
(TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/reward/best@8/std': "
(TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/reward/worst@8/mean': "
(TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/reward/worst@8/std': "
(TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/reward/maj@8/mean': "
(TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/reward/maj@8/std': "
(TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-aux/sciknoweval/reward/best@16/mean': "
(TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-aux/sciknoweval/reward/best@16/std': "
(TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/reward/worst@16/mean': "
(TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/reward/worst@16/std': "
(TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-aux/sciknoweval/reward/maj@16/mean': "
(TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-aux/sciknoweval/reward/maj@16/std': "
(TaskRunner pid=2026458) "np.float64(0.03834834240388913), 'val-aux/sciknoweval/score/mean@16': "
(TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/score/std@16': "
(TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/score/best@2/mean': "
(TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/score/best@2/std': "
(TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/score/worst@2/mean': "
(TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/score/worst@2/std': "
(TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/score/maj@2/mean': "
(TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/score/maj@2/std': "
(TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/score/best@4/mean': "
(TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/score/best@4/std': "
(TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/score/worst@4/mean': "
(TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/score/worst@4/std': "
(TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/score/maj@4/mean': "
(TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/score/maj@4/std': "
(TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/score/best@8/mean': "
(TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/score/best@8/std': "
(TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/score/worst@8/mean': "
(TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/score/worst@8/std': "
(TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/score/maj@8/mean': "
(TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/score/maj@8/std': "
(TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-aux/sciknoweval/score/best@16/mean': "
(TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-aux/sciknoweval/score/best@16/std': "
(TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/score/worst@16/mean': "
(TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/score/worst@16/std': "
(TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-aux/sciknoweval/score/maj@16/mean': "
(TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-aux/sciknoweval/score/maj@16/std': "
(TaskRunner pid=2026458) "np.float64(0.03834834240388913), 'val-core/sciknoweval/acc/mean@16': "
(TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/acc/std@16': "
(TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/acc/best@2/mean': "
(TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/acc/best@2/std': "
(TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/acc/worst@2/mean': "
(TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/acc/worst@2/std': "
(TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/acc/maj@2/mean': "
(TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/acc/maj@2/std': "
(TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/acc/best@4/mean': "
(TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/acc/best@4/std': "
(TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/acc/worst@4/mean': "
(TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/acc/worst@4/std': "
(TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/acc/maj@4/mean': "
(TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/acc/maj@4/std': "
(TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/acc/best@8/mean': "
(TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/acc/best@8/std': "
(TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/acc/worst@8/mean': "
(TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/acc/worst@8/std': "
(TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/acc/maj@8/mean': "
(TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/acc/maj@8/std': "
(TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-core/sciknoweval/acc/best@16/mean': "
(TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-core/sciknoweval/acc/best@16/std': "
(TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/acc/worst@16/mean': "
(TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/acc/worst@16/std': "
(TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-core/sciknoweval/acc/maj@16/mean': "
(TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-core/sciknoweval/acc/maj@16/std': "
(TaskRunner pid=2026458) 'np.float64(0.03834834240388913), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/mean@16': "
(TaskRunner pid=2026458) 'np.float64(0.999702380952381), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/std@16': "
(TaskRunner pid=2026458) 'np.float64(0.0011526736149426837), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@2/mean': "
(TaskRunner pid=2026458) 'np.float64(0.9999904761904761), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@2/std': "
(TaskRunner pid=2026458) 'np.float64(0.0002127457895589398), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': "
(TaskRunner pid=2026458) 'np.float64(0.9994476190476191), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@2/std': "
(TaskRunner pid=2026458) 'np.float64(0.0015248805759714617), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9997), "
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@2/std': "
(TaskRunner pid=2026458) 'np.float64(0.0011569664768572288), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), "
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), "
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': "
(TaskRunner pid=2026458) 'np.float64(0.9988761904761905), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@4/std': "
(TaskRunner pid=2026458) 'np.float64(0.0020220104047945714), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': "
(TaskRunner pid=2026458) 'np.float64(0.9999476190476191), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@4/std': "
(TaskRunner pid=2026458) 'np.float64(0.0004966783087710074), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), "
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), "
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': "
(TaskRunner pid=2026458) 'np.float64(0.9980238095238095), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@8/std': "
(TaskRunner pid=2026458) 'np.float64(0.002346295386522406), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), "
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), "
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), "
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), "
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': "
(TaskRunner pid=2026458) 'np.float64(0.9967095238095238), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@16/std': "
(TaskRunner pid=2026458) 'np.float64(0.0022003864842959084), '
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), "
(TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), "
(TaskRunner pid=2026458) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), "
(TaskRunner pid=2026458) "'val-aux/num_turns/mean': np.float64(2.0)}")
(TaskRunner pid=2026458) Training Progress: 100%|██████████| 100/100 [1:11:10<00:00, 65.49s/it] Training Progress: 100%|██████████| 100/100 [1:11:10<00:00, 42.70s/it]
(TaskRunner pid=2026458) wandb: updating run metadata
(TaskRunner pid=2026458) wandb: uploading output.log; uploading wandb-summary.json
(TaskRunner pid=2026458) wandb: uploading output.log
(TaskRunner pid=2026458) wandb: uploading history steps 98-99, summary, console lines 651-774
(TaskRunner pid=2026458) wandb:
(TaskRunner pid=2026458) wandb: Run history:
(TaskRunner pid=2026458) wandb: actor/entropy ▂▁▂▅▄█▅▅▃▆▆▆▇█▆▅▆▆▄▆▃▂▄▅▄▃▃▄▃▄▃▃▄▂▂▂▂▂▃▂
(TaskRunner pid=2026458) wandb: actor/grad_norm ▆▄▆▆▆▄▆▅▄▅▆▇▅▄▄▄▃▄▃▃▄▃█▃▄▄▃▁▂▃▃▂▂▃▃▁▃▃▂▂
(TaskRunner pid=2026458) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
(TaskRunner pid=2026458) wandb: actor/lr ▁▃▄▅▅▇██████████████████████████████████
(TaskRunner pid=2026458) wandb: actor/pg_clipfrac ▁▅▆▆▅▅▄▅▃█▃▆▄▄▆▃▃▆▃▃▃▅▃▇▆▄▃▄▄▄▅▃▃▄▅▂▄▂▅▂
(TaskRunner pid=2026458) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▅▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▁▁▁▁▁▁▁
(TaskRunner pid=2026458) wandb: actor/pg_loss █▅▅▆▆▄▇▅▄▄▃▃▃▄▁▂▃▂▄▃▃▃▃▂▃▂▃▂▂▂▃▃▃▄▄▁▂▂▂▂
(TaskRunner pid=2026458) wandb: actor/ppo_kl ▃▄▁▃▆▂▅▃▄▂▁▃▃█▆▆▄▄▅▅▃▅▆▂▆▆▆▂█▄▃▄▆▄▅▆█▄▇▄
(TaskRunner pid=2026458) wandb: critic/advantages/max █████████▃██▆████████████████▆██▆▆▆▁████
(TaskRunner pid=2026458) wandb: critic/advantages/mean ▃█▅▄▃▂▃▄▁▂▃▄▄▁▄▃▄▂▃▅▄▄▃▄▃▂▄▃▃▄▄▂▄▁▄▃▃▄▂▃
(TaskRunner pid=2026458) wandb: +220 ...
(TaskRunner pid=2026458) wandb:
(TaskRunner pid=2026458) wandb: Run summary:
(TaskRunner pid=2026458) wandb: actor/entropy 0.21838
(TaskRunner pid=2026458) wandb: actor/grad_norm 0.25845
(TaskRunner pid=2026458) wandb: actor/kl_loss 0
(TaskRunner pid=2026458) wandb: actor/lr 0.0
(TaskRunner pid=2026458) wandb: actor/pg_clipfrac 0.00042
(TaskRunner pid=2026458) wandb: actor/pg_clipfrac_lower 0
(TaskRunner pid=2026458) wandb: actor/pg_loss 0.00233
(TaskRunner pid=2026458) wandb: actor/ppo_kl 0.00011
(TaskRunner pid=2026458) wandb: critic/advantages/max 0.75
(TaskRunner pid=2026458) wandb: critic/advantages/mean -0.00115
(TaskRunner pid=2026458) wandb: +220 ...
(TaskRunner pid=2026458) wandb:
(TaskRunner pid=2026458) wandb: 🚀 View run qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/bz6p2yxy
(TaskRunner pid=2026458) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root
(TaskRunner pid=2026458) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
(TaskRunner pid=2026458) wandb: Find logs at: ./wandb/run-20260702_062054-bz6p2yxy/logs
(TaskRunner pid=2026458) Exception ignored in atexit callback: <function _start_and_connect_service.<locals>.teardown_atexit at 0x7a92e933fb00>
(TaskRunner pid=2026458) Traceback (most recent call last):
(TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit
(TaskRunner pid=2026458) conn.teardown(hooks.exit_code)
(TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown
(TaskRunner pid=2026458) self._asyncer.run(publish_teardown_and_close)
(TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run
(TaskRunner pid=2026458) return future.result()
(TaskRunner pid=2026458) ^^^^^^^^^^^^^^^
(TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result
(TaskRunner pid=2026458) return self.__get_result()
(TaskRunner pid=2026458) ^^^^^^^^^^^^^^^^^^^
(TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
(TaskRunner pid=2026458) raise self._exception
(TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap
(TaskRunner pid=2026458) return await fn()
(TaskRunner pid=2026458) ^^^^^^^^^^
(TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close
(TaskRunner pid=2026458) await self._client.close()
(TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close
(TaskRunner pid=2026458) await self._writer.wait_closed()
(TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed
(TaskRunner pid=2026458) await self._protocol._get_close_waiter(self)
(TaskRunner pid=2026458) BrokenPipeError: [Errno 32] Broken pipe
main_ppo exit code: 0
[2026-07-02 07:32:12] Finished chemistry rlsd_tr
[2026-07-02 07:32:12] Starting physics grpo
Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8
Dataset: physics
Method: grpo
Config: baseline_grpo
Model: Qwen/Qwen3-4B
Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet
Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet
Ray tmp: /tmp/ray_q3g_physics_grpo_Qwen3_4B
2026-07-02 07:32:14,409 INFO scripts.py:1364 -- Did not find any active Ray processes.
Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8
Config: baseline_grpo
Task: datasets/sciknoweval/physics
Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0
ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}}
2026-07-02 07:32:24,172 INFO worker.py:2007 -- Started a local Ray instance.
/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
(TaskRunner pid=2049544) TaskRunner hostname: mole-workspace-rw, PID: 2049544
(TaskRunner pid=2049544) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2049544) 'calculate_entropy': False,
(TaskRunner pid=2049544) 'calculate_sum_pi_squared': False,
(TaskRunner pid=2049544) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2049544) 'async_save': False,
(TaskRunner pid=2049544) 'load_contents': ['model',
(TaskRunner pid=2049544) 'optimizer',
(TaskRunner pid=2049544) 'extra'],
(TaskRunner pid=2049544) 'save_contents': ['model',
(TaskRunner pid=2049544) 'optimizer',
(TaskRunner pid=2049544) 'extra']},
(TaskRunner pid=2049544) 'clip_ratio': 0.2,
(TaskRunner pid=2049544) 'clip_ratio_c': 3.0,
(TaskRunner pid=2049544) 'clip_ratio_high': 0.28,
(TaskRunner pid=2049544) 'clip_ratio_low': 0.2,
(TaskRunner pid=2049544) 'data_loader_seed': 42,
(TaskRunner pid=2049544) 'entropy_checkpointing': False,
(TaskRunner pid=2049544) 'entropy_coeff': 0,
(TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2049544) 'freeze_vision_tower': False,
(TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2049544) 'dtype': 'bfloat16',
(TaskRunner pid=2049544) 'entropy_checkpointing': False,
(TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2049544) 'forward_only': False,
(TaskRunner pid=2049544) 'forward_prefetch': False,
(TaskRunner pid=2049544) 'fsdp_size': -1,
(TaskRunner pid=2049544) 'full_determinism': False,
(TaskRunner pid=2049544) 'model_dtype': 'fp32',
(TaskRunner pid=2049544) 'offload_policy': False,
(TaskRunner pid=2049544) 'optimizer_offload': False,
(TaskRunner pid=2049544) 'param_offload': False,
(TaskRunner pid=2049544) 'reshard_after_forward': True,
(TaskRunner pid=2049544) 'seed': 42,
(TaskRunner pid=2049544) 'strategy': 'fsdp',
(TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2049544) 'use_orig_params': False,
(TaskRunner pid=2049544) 'use_torch_compile': True,
(TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2049544) 'grad_clip': 1.0,
(TaskRunner pid=2049544) 'kl_loss_coef': 0.0,
(TaskRunner pid=2049544) 'kl_loss_type': 'low_var_kl',
(TaskRunner pid=2049544) 'loss_agg_mode': 'token-mean',
(TaskRunner pid=2049544) 'loss_scale_factor': None,
(TaskRunner pid=2049544) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2049544) 'betas': [0.9, 0.999],
(TaskRunner pid=2049544) 'clip_grad': 1.0,
(TaskRunner pid=2049544) 'lr': 1e-06,
(TaskRunner pid=2049544) 'lr_scheduler_type': 'constant',
(TaskRunner pid=2049544) 'lr_warmup_steps': 10,
(TaskRunner pid=2049544) 'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2049544) 'min_lr_ratio': 0.0,
(TaskRunner pid=2049544) 'num_cycles': 0.5,
(TaskRunner pid=2049544) 'optimizer': 'AdamW',
(TaskRunner pid=2049544) 'optimizer_impl': 'torch.optim',
(TaskRunner pid=2049544) 'override_optimizer_config': None,
(TaskRunner pid=2049544) 'total_training_steps': -1,
(TaskRunner pid=2049544) 'warmup_style': None,
(TaskRunner pid=2049544) 'weight_decay': 0.01},
(TaskRunner pid=2049544) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig',
(TaskRunner pid=2049544) 'clip_cov_lb': 1.0,
(TaskRunner pid=2049544) 'clip_cov_ratio': 0.0002,
(TaskRunner pid=2049544) 'clip_cov_ub': 5.0,
(TaskRunner pid=2049544) 'kl_cov_ratio': 0.0002,
(TaskRunner pid=2049544) 'loss_mode': 'vanilla',
(TaskRunner pid=2049544) 'ppo_kl_coef': 0.1},
(TaskRunner pid=2049544) 'ppo_epochs': 1,
(TaskRunner pid=2049544) 'ppo_max_token_len_per_gpu': 10240,
(TaskRunner pid=2049544) 'ppo_micro_batch_size': None,
(TaskRunner pid=2049544) 'ppo_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2049544) 'ppo_mini_batch_size': 8,
(TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2049544) 'all_ranks': False,
(TaskRunner pid=2049544) 'enable': False,
(TaskRunner pid=2049544) 'ranks': [],
(TaskRunner pid=2049544) 'save_path': 'outputs/profile',
(TaskRunner pid=2049544) 'tool': None,
(TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2049544) 'analysis': True,
(TaskRunner pid=2049544) 'contents': [],
(TaskRunner pid=2049544) 'discrete': False,
(TaskRunner pid=2049544) 'level': 'level0'},
(TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2049544) 'discrete': False},
(TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2049544) 'step_end': None,
(TaskRunner pid=2049544) 'step_start': 0},
(TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2049544) 'stack_depth': 32,
(TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2049544) 'rollout_n': 8,
(TaskRunner pid=2049544) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2049544) 'mode': 'disabled',
(TaskRunner pid=2049544) 'record_file': None,
(TaskRunner pid=2049544) 'replay_file': None},
(TaskRunner pid=2049544) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig',
(TaskRunner pid=2049544) 'alpha': 0.5,
(TaskRunner pid=2049544) 'distillation_add_tail': True,
(TaskRunner pid=2049544) 'distillation_topk': 100,
(TaskRunner pid=2049544) 'dont_reprompt_on_self_success': True,
(TaskRunner pid=2049544) 'environment_feedback_only_without_solution': True,
(TaskRunner pid=2049544) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig',
(TaskRunner pid=2049544) 'enable': False,
(TaskRunner pid=2049544) 'loss_weight': 0.0,
(TaskRunner pid=2049544) 'mask': 'all'},
(TaskRunner pid=2049544) 'feedback_template': '\n'
(TaskRunner pid=2049544) 'The '
(TaskRunner pid=2049544) 'following '
(TaskRunner pid=2049544) 'is '
(TaskRunner pid=2049544) 'feedback '
(TaskRunner pid=2049544) 'from '
(TaskRunner pid=2049544) 'your '
(TaskRunner pid=2049544) 'unsuccessful '
(TaskRunner pid=2049544) 'earlier '
(TaskRunner pid=2049544) 'attempt:\n'
(TaskRunner pid=2049544) '\n'
(TaskRunner pid=2049544) '{feedback_raw}',
(TaskRunner pid=2049544) 'full_logit_distillation': True,
(TaskRunner pid=2049544) 'include_environment_feedback': True,
(TaskRunner pid=2049544) 'is_clip': 2,
(TaskRunner pid=2049544) 'max_reprompt_len': 22528,
(TaskRunner pid=2049544) 'remove_thinking_from_demonstration': False,
(TaskRunner pid=2049544) 'reprompt_template': '{prompt}{solution}{feedback}\n'
(TaskRunner pid=2049544) '\n'
(TaskRunner pid=2049544) 'Correctly '
(TaskRunner pid=2049544) 'solve '
(TaskRunner pid=2049544) 'the '
(TaskRunner pid=2049544) 'original '
(TaskRunner pid=2049544) 'question.',
(TaskRunner pid=2049544) 'reprompt_truncation': 'right',
(TaskRunner pid=2049544) 'solution_template': '\n'
(TaskRunner pid=2049544) 'Correct '
(TaskRunner pid=2049544) 'solution:\n'
(TaskRunner pid=2049544) '\n'
(TaskRunner pid=2049544) '{successful_previous_attempt}',
(TaskRunner pid=2049544) 'srpo_dynamic_weighting': False,
(TaskRunner pid=2049544) 'srpo_dynamic_weighting_temperature': 1.0,
(TaskRunner pid=2049544) 'success_reward_threshold': 0.5,
(TaskRunner pid=2049544) 'teacher_regularization': 'ema',
(TaskRunner pid=2049544) 'teacher_update_rate': 0.0,
(TaskRunner pid=2049544) 'token_reweight_decay_steps': None,
(TaskRunner pid=2049544) 'token_reweight_eps_w': 0.2,
(TaskRunner pid=2049544) 'token_reweight_lambda': 0.5},
(TaskRunner pid=2049544) 'shuffle': False,
(TaskRunner pid=2049544) 'strategy': 'fsdp',
(TaskRunner pid=2049544) 'sum_pi_squared_checkpointing': False,
(TaskRunner pid=2049544) 'tau_neg': 1.05,
(TaskRunner pid=2049544) 'tau_pos': 1.0,
(TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2049544) 'use_dynamic_bsz': False,
(TaskRunner pid=2049544) 'use_fused_kernels': False,
(TaskRunner pid=2049544) 'use_kl_loss': False,
(TaskRunner pid=2049544) 'use_prefix_grouper': False,
(TaskRunner pid=2049544) 'use_remove_padding': True,
(TaskRunner pid=2049544) 'use_torch_compile': True},
(TaskRunner pid=2049544) 'hybrid_engine': True,
(TaskRunner pid=2049544) 'model': {'_target_': 'verl.workers.config.HFModelConfig',
(TaskRunner pid=2049544) 'custom_chat_template': None,
(TaskRunner pid=2049544) 'enable_activation_offload': False,
(TaskRunner pid=2049544) 'enable_gradient_checkpointing': True,
(TaskRunner pid=2049544) 'exclude_modules': None,
(TaskRunner pid=2049544) 'external_lib': None,
(TaskRunner pid=2049544) 'fused_kernel_options': {'impl_backend': 'torch'},
(TaskRunner pid=2049544) 'hf_config_path': None,
(TaskRunner pid=2049544) 'lora_adapter_path': None,
(TaskRunner pid=2049544) 'lora_alpha': 16,
(TaskRunner pid=2049544) 'lora_rank': 0,
(TaskRunner pid=2049544) 'override_config': {},
(TaskRunner pid=2049544) 'path': 'Qwen/Qwen3-4B',
(TaskRunner pid=2049544) 'target_modules': 'all-linear',
(TaskRunner pid=2049544) 'tiled_mlp': {'enabled': False,
(TaskRunner pid=2049544) 'num_shards': 4},
(TaskRunner pid=2049544) 'tokenizer_path': None,
(TaskRunner pid=2049544) 'trust_remote_code': True,
(TaskRunner pid=2049544) 'use_fused_kernels': False,
(TaskRunner pid=2049544) 'use_liger': False,
(TaskRunner pid=2049544) 'use_remove_padding': True,
(TaskRunner pid=2049544) 'use_shm': False},
(TaskRunner pid=2049544) 'nccl_timeout': 600,
(TaskRunner pid=2049544) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2049544) 'entropy_checkpointing': False,
(TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2049544) 'dtype': 'bfloat16',
(TaskRunner pid=2049544) 'entropy_checkpointing': False,
(TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2049544) 'forward_only': True,
(TaskRunner pid=2049544) 'forward_prefetch': False,
(TaskRunner pid=2049544) 'fsdp_size': -1,
(TaskRunner pid=2049544) 'full_determinism': False,
(TaskRunner pid=2049544) 'model_dtype': 'fp32',
(TaskRunner pid=2049544) 'offload_policy': False,
(TaskRunner pid=2049544) 'optimizer_offload': False,
(TaskRunner pid=2049544) 'param_offload': False,
(TaskRunner pid=2049544) 'reshard_after_forward': True,
(TaskRunner pid=2049544) 'seed': 42,
(TaskRunner pid=2049544) 'strategy': 'fsdp',
(TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2049544) 'use_orig_params': False,
(TaskRunner pid=2049544) 'use_torch_compile': True,
(TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2049544) 'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2049544) 'log_prob_micro_batch_size': None,
(TaskRunner pid=2049544) 'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2049544) 'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2049544) 'all_ranks': False,
(TaskRunner pid=2049544) 'enable': False,
(TaskRunner pid=2049544) 'ranks': [],
(TaskRunner pid=2049544) 'save_path': 'outputs/profile',
(TaskRunner pid=2049544) 'tool': None,
(TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2049544) 'analysis': True,
(TaskRunner pid=2049544) 'contents': [],
(TaskRunner pid=2049544) 'discrete': False,
(TaskRunner pid=2049544) 'level': 'level0'},
(TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2049544) 'discrete': False},
(TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2049544) 'step_end': None,
(TaskRunner pid=2049544) 'step_start': 0},
(TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2049544) 'stack_depth': 32,
(TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2049544) 'rollout_n': 8,
(TaskRunner pid=2049544) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2049544) 'mode': 'disabled',
(TaskRunner pid=2049544) 'record_file': None,
(TaskRunner pid=2049544) 'replay_file': None},
(TaskRunner pid=2049544) 'strategy': 'fsdp',
(TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2049544) 'use_torch_compile': True},
(TaskRunner pid=2049544) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2049544) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig',
(TaskRunner pid=2049544) 'agent_loop_config_path': None,
(TaskRunner pid=2049544) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig',
(TaskRunner pid=2049544) 'name': None,
(TaskRunner pid=2049544) 'path': None},
(TaskRunner pid=2049544) 'default_agent_loop': 'single_turn_agent',
(TaskRunner pid=2049544) 'num_workers': 8},
(TaskRunner pid=2049544) 'calculate_log_probs': True,
(TaskRunner pid=2049544) 'cudagraph_capture_sizes': None,
(TaskRunner pid=2049544) 'data_parallel_size': 1,
(TaskRunner pid=2049544) 'disable_log_stats': True,
(TaskRunner pid=2049544) 'do_sample': True,
(TaskRunner pid=2049544) 'dtype': 'bfloat16',
(TaskRunner pid=2049544) 'enable_chunked_prefill': True,
(TaskRunner pid=2049544) 'enable_prefix_caching': True,
(TaskRunner pid=2049544) 'enable_rollout_routing_replay': False,
(TaskRunner pid=2049544) 'enforce_eager': False,
(TaskRunner pid=2049544) 'engine_kwargs': {'sglang': {}, 'vllm': {}},
(TaskRunner pid=2049544) 'expert_parallel_size': 1,
(TaskRunner pid=2049544) 'free_cache_engine': True,
(TaskRunner pid=2049544) 'gpu_memory_utilization': 0.8,
(TaskRunner pid=2049544) 'ignore_eos': False,
(TaskRunner pid=2049544) 'layered_summon': False,
(TaskRunner pid=2049544) 'load_format': 'dummy',
(TaskRunner pid=2049544) 'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2049544) 'log_prob_micro_batch_size': None,
(TaskRunner pid=2049544) 'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2049544) 'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2049544) 'logprobs_mode': 'processed_logprobs',
(TaskRunner pid=2049544) 'max_model_len': 10240,
(TaskRunner pid=2049544) 'max_num_batched_tokens': 10240,
(TaskRunner pid=2049544) 'max_num_seqs': 1024,
(TaskRunner pid=2049544) 'mode': 'async',
(TaskRunner pid=2049544) 'multi_stage_wake_up': False,
(TaskRunner pid=2049544) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig',
(TaskRunner pid=2049544) 'enable': False,
(TaskRunner pid=2049544) 'format': 'hermes',
(TaskRunner pid=2049544) 'interaction_config_path': None,
(TaskRunner pid=2049544) 'max_assistant_turns': None,
(TaskRunner pid=2049544) 'max_parallel_calls': 1,
(TaskRunner pid=2049544) 'max_tool_response_length': 256,
(TaskRunner pid=2049544) 'max_user_turns': None,
(TaskRunner pid=2049544) 'num_repeat_rollouts': None,
(TaskRunner pid=2049544) 'tokenization_sanity_check_mode': 'strict',
(TaskRunner pid=2049544) 'tool_config_path': None,
(TaskRunner pid=2049544) 'tool_response_truncate_side': 'middle',
(TaskRunner pid=2049544) 'use_inference_chat_template': False},
(TaskRunner pid=2049544) 'n': 8,
(TaskRunner pid=2049544) 'name': 'vllm',
(TaskRunner pid=2049544) 'over_sample_rate': 0,
(TaskRunner pid=2049544) 'pipeline_model_parallel_size': 1,
(TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2049544) 'all_ranks': False,
(TaskRunner pid=2049544) 'enable': False,
(TaskRunner pid=2049544) 'ranks': [],
(TaskRunner pid=2049544) 'save_path': 'outputs/profile',
(TaskRunner pid=2049544) 'tool': None,
(TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2049544) 'analysis': True,
(TaskRunner pid=2049544) 'contents': [],
(TaskRunner pid=2049544) 'discrete': False,
(TaskRunner pid=2049544) 'level': 'level0'},
(TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2049544) 'discrete': False},
(TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2049544) 'step_end': None,
(TaskRunner pid=2049544) 'step_start': 0},
(TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2049544) 'stack_depth': 32,
(TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2049544) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig',
(TaskRunner pid=2049544) 'enable': False,
(TaskRunner pid=2049544) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml',
(TaskRunner pid=2049544) 'port': 9090,
(TaskRunner pid=2049544) 'served_model_name': 'Qwen/Qwen3-4B'},
(TaskRunner pid=2049544) 'prompt_length': 2048,
(TaskRunner pid=2049544) 'quantization': None,
(TaskRunner pid=2049544) 'quantization_config_file': None,
(TaskRunner pid=2049544) 'response_length': 8192,
(TaskRunner pid=2049544) 'scheduling_policy': 'fcfs',
(TaskRunner pid=2049544) 'skip_dump_dir': '/tmp/rollout_dump',
(TaskRunner pid=2049544) 'skip_rollout': False,
(TaskRunner pid=2049544) 'skip_tokenizer_init': True,
(TaskRunner pid=2049544) 'temperature': 1.0,
(TaskRunner pid=2049544) 'tensor_model_parallel_size': 2,
(TaskRunner pid=2049544) 'top_k': -1,
(TaskRunner pid=2049544) 'top_p': 1.0,
(TaskRunner pid=2049544) 'trace': {'_target_': 'verl.workers.config.TraceConfig',
(TaskRunner pid=2049544) 'backend': None,
(TaskRunner pid=2049544) 'max_samples_per_step_per_worker': None,
(TaskRunner pid=2049544) 'token2text': False},
(TaskRunner pid=2049544) 'update_weights_bucket_megabytes': 512,
(TaskRunner pid=2049544) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig',
(TaskRunner pid=2049544) 'do_sample': True,
(TaskRunner pid=2049544) 'n': 16,
(TaskRunner pid=2049544) 'temperature': 0.6,
(TaskRunner pid=2049544) 'top_k': -1,
(TaskRunner pid=2049544) 'top_p': 0.95}}},
(TaskRunner pid=2049544) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig',
(TaskRunner pid=2049544) 'adv_estimator': 'grpo',
(TaskRunner pid=2049544) 'gamma': 1.0,
(TaskRunner pid=2049544) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig',
(TaskRunner pid=2049544) 'horizon': 10000,
(TaskRunner pid=2049544) 'kl_coef': 0.001,
(TaskRunner pid=2049544) 'target_kl': 0.1,
(TaskRunner pid=2049544) 'type': 'fixed'},
(TaskRunner pid=2049544) 'kl_penalty': 'kl',
(TaskRunner pid=2049544) 'lam': 1.0,
(TaskRunner pid=2049544) 'norm_adv_by_std_in_grpo': False,
(TaskRunner pid=2049544) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0},
(TaskRunner pid=2049544) 'rollout_correction': {'bypass_mode': False,
(TaskRunner pid=2049544) 'loss_type': 'ppo_clip',
(TaskRunner pid=2049544) 'rollout_is': 'token',
(TaskRunner pid=2049544) 'rollout_is_batch_normalize': False,
(TaskRunner pid=2049544) 'rollout_is_threshold': 2.0,
(TaskRunner pid=2049544) 'rollout_rs': None,
(TaskRunner pid=2049544) 'rollout_rs_threshold': None},
(TaskRunner pid=2049544) 'use_kl_in_reward': False,
(TaskRunner pid=2049544) 'use_pf_ppo': False},
(TaskRunner pid=2049544) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig',
(TaskRunner pid=2049544) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2049544) 'async_save': False,
(TaskRunner pid=2049544) 'load_contents': ['model', 'optimizer', 'extra'],
(TaskRunner pid=2049544) 'save_contents': ['model', 'optimizer', 'extra']},
(TaskRunner pid=2049544) 'cliprange_value': 0.5,
(TaskRunner pid=2049544) 'data_loader_seed': 42,
(TaskRunner pid=2049544) 'enable': None,
(TaskRunner pid=2049544) 'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2049544) 'forward_micro_batch_size': None,
(TaskRunner pid=2049544) 'forward_micro_batch_size_per_gpu': None,
(TaskRunner pid=2049544) 'grad_clip': 1.0,
(TaskRunner pid=2049544) 'loss_agg_mode': 'token-mean',
(TaskRunner pid=2049544) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg',
(TaskRunner pid=2049544) 'enable_activation_offload': False,
(TaskRunner pid=2049544) 'enable_gradient_checkpointing': True,
(TaskRunner pid=2049544) 'external_lib': None,
(TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2049544) 'dtype': 'bfloat16',
(TaskRunner pid=2049544) 'entropy_checkpointing': False,
(TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2049544) 'forward_only': False,
(TaskRunner pid=2049544) 'forward_prefetch': False,
(TaskRunner pid=2049544) 'fsdp_size': -1,
(TaskRunner pid=2049544) 'full_determinism': False,
(TaskRunner pid=2049544) 'model_dtype': 'fp32',
(TaskRunner pid=2049544) 'offload_policy': False,
(TaskRunner pid=2049544) 'optimizer_offload': False,
(TaskRunner pid=2049544) 'param_offload': False,
(TaskRunner pid=2049544) 'reshard_after_forward': True,
(TaskRunner pid=2049544) 'seed': 42,
(TaskRunner pid=2049544) 'strategy': 'fsdp',
(TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2049544) 'use_orig_params': False,
(TaskRunner pid=2049544) 'use_torch_compile': True,
(TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2049544) 'lora_alpha': 16,
(TaskRunner pid=2049544) 'lora_rank': 0,
(TaskRunner pid=2049544) 'override_config': {},
(TaskRunner pid=2049544) 'path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2049544) 'target_modules': 'all-linear',
(TaskRunner pid=2049544) 'tiled_mlp': {'enabled': False, 'num_shards': 4},
(TaskRunner pid=2049544) 'tokenizer_path': 'Qwen/Qwen3-4B',
(TaskRunner pid=2049544) 'trust_remote_code': True,
(TaskRunner pid=2049544) 'use_remove_padding': False,
(TaskRunner pid=2049544) 'use_shm': False},
(TaskRunner pid=2049544) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2049544) 'betas': [0.9, 0.999],
(TaskRunner pid=2049544) 'clip_grad': 1.0,
(TaskRunner pid=2049544) 'lr': 1e-05,
(TaskRunner pid=2049544) 'lr_scheduler_type': 'constant',
(TaskRunner pid=2049544) 'lr_warmup_steps': -1,
(TaskRunner pid=2049544) 'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2049544) 'min_lr_ratio': 0.0,
(TaskRunner pid=2049544) 'num_cycles': 0.5,
(TaskRunner pid=2049544) 'optimizer': 'AdamW',
(TaskRunner pid=2049544) 'optimizer_impl': 'torch.optim',
(TaskRunner pid=2049544) 'override_optimizer_config': None,
(TaskRunner pid=2049544) 'total_training_steps': -1,
(TaskRunner pid=2049544) 'warmup_style': None,
(TaskRunner pid=2049544) 'weight_decay': 0.01},
(TaskRunner pid=2049544) 'ppo_epochs': 1,
(TaskRunner pid=2049544) 'ppo_max_token_len_per_gpu': 32768,
(TaskRunner pid=2049544) 'ppo_micro_batch_size': None,
(TaskRunner pid=2049544) 'ppo_micro_batch_size_per_gpu': None,
(TaskRunner pid=2049544) 'ppo_mini_batch_size': 8,
(TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2049544) 'all_ranks': False,
(TaskRunner pid=2049544) 'enable': False,
(TaskRunner pid=2049544) 'ranks': [],
(TaskRunner pid=2049544) 'save_path': 'outputs/profile',
(TaskRunner pid=2049544) 'tool': None,
(TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2049544) 'analysis': True,
(TaskRunner pid=2049544) 'contents': [],
(TaskRunner pid=2049544) 'discrete': False,
(TaskRunner pid=2049544) 'level': 'level0'},
(TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2049544) 'discrete': False},
(TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2049544) 'step_end': None,
(TaskRunner pid=2049544) 'step_start': 0},
(TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2049544) 'stack_depth': 32,
(TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2049544) 'rollout_n': 8,
(TaskRunner pid=2049544) 'shuffle': False,
(TaskRunner pid=2049544) 'strategy': 'fsdp',
(TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2049544) 'use_dynamic_bsz': False},
(TaskRunner pid=2049544) 'custom_reward_function': {'name': 'compute_score',
(TaskRunner pid=2049544) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'},
(TaskRunner pid=2049544) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False},
(TaskRunner pid=2049544) 'custom_cls': {'name': None, 'path': None},
(TaskRunner pid=2049544) 'datagen': {'name': None, 'path': None},
(TaskRunner pid=2049544) 'dataloader_num_workers': 8,
(TaskRunner pid=2049544) 'filter_overlong_prompts': True,
(TaskRunner pid=2049544) 'filter_overlong_prompts_workers': 1,
(TaskRunner pid=2049544) 'image_key': 'images',
(TaskRunner pid=2049544) 'image_patch_size': 14,
(TaskRunner pid=2049544) 'max_prompt_length': 2048,
(TaskRunner pid=2049544) 'max_response_length': 8192,
(TaskRunner pid=2049544) 'prompt_key': 'prompt',
(TaskRunner pid=2049544) 'return_full_prompt': False,
(TaskRunner pid=2049544) 'return_multi_modal_inputs': True,
(TaskRunner pid=2049544) 'return_raw_chat': True,
(TaskRunner pid=2049544) 'return_raw_input_ids': False,
(TaskRunner pid=2049544) 'reward_fn_key': 'data_source',
(TaskRunner pid=2049544) 'sampler': {'class_name': None, 'class_path': None},
(TaskRunner pid=2049544) 'seed': None,
(TaskRunner pid=2049544) 'shuffle': True,
(TaskRunner pid=2049544) 'tokenizer': None,
(TaskRunner pid=2049544) 'tool_config_path': None,
(TaskRunner pid=2049544) 'train_batch_size': 32,
(TaskRunner pid=2049544) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'],
(TaskRunner pid=2049544) 'train_max_samples': 3200,
(TaskRunner pid=2049544) 'truncation': 'error',
(TaskRunner pid=2049544) 'trust_remote_code': True,
(TaskRunner pid=2049544) 'use_shm': False,
(TaskRunner pid=2049544) 'val_batch_size': None,
(TaskRunner pid=2049544) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'],
(TaskRunner pid=2049544) 'val_max_samples': -1,
(TaskRunner pid=2049544) 'validation_shuffle': False,
(TaskRunner pid=2049544) 'video_key': 'videos'},
(TaskRunner pid=2049544) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2049544) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2049544) 'controller_nsight_options': {'cuda-graph-trace': 'graph',
(TaskRunner pid=2049544) 'cuda-memory-usage': 'true',
(TaskRunner pid=2049544) 'trace': 'cuda,nvtx,cublas,ucx'},
(TaskRunner pid=2049544) 'discrete': False,
(TaskRunner pid=2049544) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi',
(TaskRunner pid=2049544) 'capture-range-end': None,
(TaskRunner pid=2049544) 'cuda-graph-trace': 'graph',
(TaskRunner pid=2049544) 'cuda-memory-usage': 'true',
(TaskRunner pid=2049544) 'kill': 'none',
(TaskRunner pid=2049544) 'trace': 'cuda,nvtx,cublas,ucx'}},
(TaskRunner pid=2049544) 'torch_memory': {'context': 'all',
(TaskRunner pid=2049544) 'kw_args': {},
(TaskRunner pid=2049544) 'stack_depth': 32,
(TaskRunner pid=2049544) 'stacks': 'all',
(TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}},
(TaskRunner pid=2049544) 'profile_continuous_steps': False,
(TaskRunner pid=2049544) 'save_path': 'outputs/profile',
(TaskRunner pid=2049544) 'steps': None,
(TaskRunner pid=2049544) 'tool': None},
(TaskRunner pid=2049544) 'max_model_len': 10240,
(TaskRunner pid=2049544) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_4B',
(TaskRunner pid=2049544) 'include_dashboard': False,
(TaskRunner pid=2049544) 'num_cpus': None,
(TaskRunner pid=2049544) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2049544) 'TASK': 'datasets/sciknoweval/physics',
(TaskRunner pid=2049544) 'USER': 'root'}}},
(TaskRunner pid=2049544) 'timeline_json_file': None},
(TaskRunner pid=2049544) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig',
(TaskRunner pid=2049544) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig',
(TaskRunner pid=2049544) 'name': 'custom_reward_manager',
(TaskRunner pid=2049544) 'path': None},
(TaskRunner pid=2049544) 'name': 'naive',
(TaskRunner pid=2049544) 'source': 'register'},
(TaskRunner pid=2049544) 'reward_model': {'enable': False,
(TaskRunner pid=2049544) 'enable_resource_pool': False,
(TaskRunner pid=2049544) 'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2049544) 'launch_reward_fn_async': False,
(TaskRunner pid=2049544) 'max_length': None,
(TaskRunner pid=2049544) 'micro_batch_size': None,
(TaskRunner pid=2049544) 'micro_batch_size_per_gpu': None,
(TaskRunner pid=2049544) 'model': {'external_lib': None,
(TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2049544) 'forward_prefetch': False,
(TaskRunner pid=2049544) 'fsdp_size': -1,
(TaskRunner pid=2049544) 'param_offload': False,
(TaskRunner pid=2049544) 'reshard_after_forward': True,
(TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2049544) 'input_tokenizer': 'Qwen/Qwen3-4B',
(TaskRunner pid=2049544) 'override_config': {},
(TaskRunner pid=2049544) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1',
(TaskRunner pid=2049544) 'trust_remote_code': False,
(TaskRunner pid=2049544) 'use_fused_kernels': False,
(TaskRunner pid=2049544) 'use_remove_padding': False,
(TaskRunner pid=2049544) 'use_shm': False},
(TaskRunner pid=2049544) 'n_gpus_per_node': 8,
(TaskRunner pid=2049544) 'nnodes': 0,
(TaskRunner pid=2049544) 'num_workers': 1,
(TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2049544) 'all_ranks': False,
(TaskRunner pid=2049544) 'enable': False,
(TaskRunner pid=2049544) 'ranks': [],
(TaskRunner pid=2049544) 'save_path': 'outputs/profile',
(TaskRunner pid=2049544) 'tool': None,
(TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2049544) 'analysis': True,
(TaskRunner pid=2049544) 'contents': [],
(TaskRunner pid=2049544) 'discrete': False,
(TaskRunner pid=2049544) 'level': 'level0'},
(TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2049544) 'discrete': False},
(TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2049544) 'step_end': None,
(TaskRunner pid=2049544) 'step_start': 0},
(TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2049544) 'stack_depth': 32,
(TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2049544) 'reward_loop_class_name': None,
(TaskRunner pid=2049544) 'reward_loop_module_path': None,
(TaskRunner pid=2049544) 'reward_loop_source': 'register',
(TaskRunner pid=2049544) 'reward_manager': 'naive',
(TaskRunner pid=2049544) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2049544) 'cudagraph_capture_sizes': None,
(TaskRunner pid=2049544) 'data_parallel_size': 1,
(TaskRunner pid=2049544) 'disable_log_stats': True,
(TaskRunner pid=2049544) 'dtype': 'bfloat16',
(TaskRunner pid=2049544) 'enable_chunked_prefill': True,
(TaskRunner pid=2049544) 'enable_prefix_caching': True,
(TaskRunner pid=2049544) 'enforce_eager': True,
(TaskRunner pid=2049544) 'engine_kwargs': {},
(TaskRunner pid=2049544) 'expert_parallel_size': 1,
(TaskRunner pid=2049544) 'free_cache_engine': True,
(TaskRunner pid=2049544) 'gpu_memory_utilization': 0.5,
(TaskRunner pid=2049544) 'limit_images': None,
(TaskRunner pid=2049544) 'load_format': 'auto',
(TaskRunner pid=2049544) 'max_model_len': None,
(TaskRunner pid=2049544) 'max_num_batched_tokens': 8192,
(TaskRunner pid=2049544) 'max_num_seqs': 1024,
(TaskRunner pid=2049544) 'name': '???',
(TaskRunner pid=2049544) 'prompt_length': 2048,
(TaskRunner pid=2049544) 'response_length': 2048,
(TaskRunner pid=2049544) 'skip_tokenizer_init': False,
(TaskRunner pid=2049544) 'tensor_model_parallel_size': 2},
(TaskRunner pid=2049544) 'sandbox_fusion': {'max_concurrent': 64,
(TaskRunner pid=2049544) 'memory_limit_mb': 1024,
(TaskRunner pid=2049544) 'url': None},
(TaskRunner pid=2049544) 'strategy': 'fsdp',
(TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2049544) 'use_dynamic_bsz': False,
(TaskRunner pid=2049544) 'use_reward_loop': False},
(TaskRunner pid=2049544) 'trainer': {'balance_batch': True,
(TaskRunner pid=2049544) 'critic_warmup': 0,
(TaskRunner pid=2049544) 'default_hdfs_dir': None,
(TaskRunner pid=2049544) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2049544) 'del_local_ckpt_after_load': False,
(TaskRunner pid=2049544) 'device': 'cuda',
(TaskRunner pid=2049544) 'esi_redundant_time': 0,
(TaskRunner pid=2049544) 'experiment_name': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2049544) 'group_name': 'QWEN3-GRPO-generalization',
(TaskRunner pid=2049544) 'log_val_generations': 0,
(TaskRunner pid=2049544) 'logger': ['console', 'wandb'],
(TaskRunner pid=2049544) 'max_actor_ckpt_to_keep': 1,
(TaskRunner pid=2049544) 'max_critic_ckpt_to_keep': None,
(TaskRunner pid=2049544) 'n_gpus_per_node': 8,
(TaskRunner pid=2049544) 'nnodes': 1,
(TaskRunner pid=2049544) 'project_name': 'SDPO-root',
(TaskRunner pid=2049544) 'ray_wait_register_center_timeout': 300,
(TaskRunner pid=2049544) 'resume_from_path': None,
(TaskRunner pid=2049544) 'resume_mode': 'auto',
(TaskRunner pid=2049544) 'rollout_data_dir': None,
(TaskRunner pid=2049544) 'save_freq': 10,
(TaskRunner pid=2049544) 'test_freq': 10,
(TaskRunner pid=2049544) 'total_epochs': 30,
(TaskRunner pid=2049544) 'total_training_steps': 100,
(TaskRunner pid=2049544) 'use_legacy_worker_impl': 'auto',
(TaskRunner pid=2049544) 'val_before_train': False,
(TaskRunner pid=2049544) 'val_only': False,
(TaskRunner pid=2049544) 'validation_data_dir': None},
(TaskRunner pid=2049544) 'transfer_queue': {'enable': False},
(TaskRunner pid=2049544) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics',
(TaskRunner pid=2049544) 'dir': '/users/root/SDPO',
(TaskRunner pid=2049544) 'log_dir': '/users/root/output',
(TaskRunner pid=2049544) 'task': 'datasets/sciknoweval/physics'}}
(TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2049544) use_critic=need_critic(config),
(TaskRunner pid=2049544) [validate_config] All configuration checks passed successfully!
(TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2049544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(TaskRunner pid=2049544) Using dataset class: RLHFDataset
(TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(TaskRunner pid=2049544) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(TaskRunner pid=2049544) dataset len: 720
(TaskRunner pid=2049544) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2049544) WARNING:2026-07-02 07:32:33,214:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00<?, ? examples/s]
(TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 720/720 [00:00<00:00, 834.24 examples/s]
(TaskRunner pid=2049544) filter dataset len: 720
(TaskRunner pid=2049544) Using dataset class: RLHFDataset
(TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 720/720 [00:00<00:00, 752.54 examples/s]
(TaskRunner pid=2049544) dataset len: 80
(TaskRunner pid=2049544) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2049544) WARNING:2026-07-02 07:32:34,593:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/80 [00:00<?, ? examples/s]
(TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 142.06 examples/s]
(TaskRunner pid=2049544) filter dataset len: 80
(TaskRunner pid=2049544) Size of train dataloader: 22, Size of val dataloader: 1
(TaskRunner pid=2049544) Total training steps: 100
(TaskRunner pid=2049544) colocated worker base class <class 'verl.single_controller.base.worker.Worker'>
(TaskRunner pid=2049544) bind role actor_rollout method chat_completion to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2049544) bind role actor_rollout method generate to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2049544) bind role actor_rollout method get_zeromq_address to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2049544) bind role actor_rollout method sleep to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2049544) bind role actor_rollout method wake_up to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 121.99 examples/s]
(TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2049544) self.use_critic = need_critic(self.config)
(WorkerDict pid=2049933) [W702 07:32:42.949021168 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59177 (errno: 97 - Address family not supported by protocol).
(WorkerDict pid=2049932) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7
(WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2049932) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2049934) [W702 07:32:42.120202714 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59177 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(WorkerDict pid=2049932) Model config after override: Qwen3Config {
(WorkerDict pid=2049932) "architectures": [
(WorkerDict pid=2049932) "Qwen3ForCausalLM"
(WorkerDict pid=2049932) ],
(WorkerDict pid=2049932) "attention_bias": false,
(WorkerDict pid=2049932) "attention_dropout": 0.0,
(WorkerDict pid=2049932) "dtype": "bfloat16",
(WorkerDict pid=2049932) "eos_token_id": 151645,
(WorkerDict pid=2049932) "head_dim": 128,
(WorkerDict pid=2049932) "hidden_act": "silu",
(WorkerDict pid=2049932) "hidden_size": 2560,
(WorkerDict pid=2049932) "initializer_range": 0.02,
(WorkerDict pid=2049932) "intermediate_size": 9728,
(WorkerDict pid=2049932) "layer_types": [
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention",
(WorkerDict pid=2049932) "full_attention"
(WorkerDict pid=2049932) ],
(WorkerDict pid=2049932) "max_position_embeddings": 40960,
(WorkerDict pid=2049932) "max_window_layers": 36,
(WorkerDict pid=2049932) "model_type": "qwen3",
(WorkerDict pid=2049932) "num_attention_heads": 32,
(WorkerDict pid=2049932) "num_hidden_layers": 36,
(WorkerDict pid=2049932) "num_key_value_heads": 8,
(WorkerDict pid=2049932) "pad_token_id": 151643,
(WorkerDict pid=2049932) "rms_norm_eps": 1e-06,
(WorkerDict pid=2049932) "rope_scaling": null,
(WorkerDict pid=2049932) "rope_theta": 1000000,
(WorkerDict pid=2049932) "sliding_window": null,
(WorkerDict pid=2049932) "tie_word_embeddings": true,
(WorkerDict pid=2049932) "transformers_version": "4.57.1",
(WorkerDict pid=2049932) "use_cache": true,
(WorkerDict pid=2049932) "use_sliding_window": false,
(WorkerDict pid=2049932) "vocab_size": 151936
(WorkerDict pid=2049932) }
(WorkerDict pid=2049932)
(WorkerDict pid=2049932) `torch_dtype` is deprecated! Use `dtype` instead!
(WorkerDict pid=2049932) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2049932) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2049932) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]
(WorkerDict pid=2049932) Loading checkpoint shards: 33%|███▎ | 1/3 [00:02<00:05, 2.87s/it]
(WorkerDict pid=2049938) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2049938) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2049938) `torch_dtype` is deprecated! Use `dtype` instead! [repeated 7x across cluster]
(WorkerDict pid=2049938) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` [repeated 14x across cluster]
(WorkerDict pid=2049938) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s] [repeated 7x across cluster]
(WorkerDict pid=2049932) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it]
(WorkerDict pid=2049935) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.91s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it]
(WorkerDict pid=2049932) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention
(WorkerDict pid=2049932) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch
(WorkerDict pid=2049938) [Gloo] Rank 6 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 [repeated 7x across cluster]
(WorkerDict pid=2049932) Qwen3ForCausalLM contains 4.02B parameters
(WorkerDict pid=2049932) wrap_policy: functools.partial(<function _or_policy at 0x7dfaf3668680>, policies=[functools.partial(<function transformer_auto_wrap_policy at 0x7dfaf3668540>, transformer_layer_cls={<class 'transformers.models.qwen3.modeling_qwen3.Qwen3DecoderLayer'>})])
(WorkerDict pid=2049932) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2049932) Total steps: 100, num_warmup_steps: 10
(WorkerDict pid=2049932) Actor use_remove_padding=True
(WorkerDict pid=2049932) Actor use_fused_kernels=False
(WorkerDict pid=2049932) Actor use_prefix_grouper=False
(WorkerDict pid=2049939) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster]
(WorkerDict pid=2049939) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster]
(WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(WorkerDict pid=2049932) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2049939) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.92s/it] [repeated 12x across cluster]
(WorkerDict pid=2049939) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 4x across cluster]
(WorkerDict pid=2049938) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.88s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.94s/it] [repeated 2x across cluster]
(WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2049932) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2049939) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster]
(WorkerDict pid=2049939) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2049932) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3
(WorkerDict pid=2049933) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
(WorkerDict pid=2049933) warnings.warn(
(WorkerDict pid=2049939) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2049939) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2049544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster]
(WorkerDict pid=2049932) warnings.warn( [repeated 7x across cluster]
(vLLMHttpServer pid=2050805) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(vLLMHttpServer pid=2050805) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(vLLMHttpServer pid=2050806) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(vLLMHttpServer pid=2050806) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(vLLMHttpServer pid=2050805) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead.
(vLLMHttpServer pid=2050805) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(vLLMHttpServer pid=2050803) ['serve',
(vLLMHttpServer pid=2050803) 'Qwen/Qwen3-4B',
(vLLMHttpServer pid=2050803) '--dtype',
(vLLMHttpServer pid=2050803) 'bfloat16',
(vLLMHttpServer pid=2050803) '--load_format',
(vLLMHttpServer pid=2050803) 'dummy',
(vLLMHttpServer pid=2050803) '--trust_remote_code',
(vLLMHttpServer pid=2050803) '--max_model_len',
(vLLMHttpServer pid=2050803) '40960',
(vLLMHttpServer pid=2050803) '--max_num_seqs',
(vLLMHttpServer pid=2050803) '1024',
(vLLMHttpServer pid=2050803) '--enable_chunked_prefill',
(vLLMHttpServer pid=2050803) '--max_num_batched_tokens',
(vLLMHttpServer pid=2050803) '10240',
(vLLMHttpServer pid=2050803) '--enable_prefix_caching',
(vLLMHttpServer pid=2050803) '--enable_sleep_mode',
(vLLMHttpServer pid=2050803) '--logprobs_mode',
(vLLMHttpServer pid=2050803) 'processed_logprobs',
(vLLMHttpServer pid=2050803) '--disable_custom_all_reduce',
(vLLMHttpServer pid=2050803) '--gpu_memory_utilization',
(vLLMHttpServer pid=2050803) '0.8',
(vLLMHttpServer pid=2050803) '--disable_log_stats',
(vLLMHttpServer pid=2050803) '--tensor_parallel_size',
(vLLMHttpServer pid=2050803) '2',
(vLLMHttpServer pid=2050803) '--seed',
(vLLMHttpServer pid=2050803) '0',
(vLLMHttpServer pid=2050803) '--override_generation_config',
(vLLMHttpServer pid=2050803) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, '
(vLLMHttpServer pid=2050803) '"max_new_tokens": 8192}',
(vLLMHttpServer pid=2050803) '--hf_overrides',
(vLLMHttpServer pid=2050803) '{}',
(vLLMHttpServer pid=2050803) '--scheduling_policy',
(vLLMHttpServer pid=2050803) 'fcfs']
(WorkerDict pid=2049938) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster]
(vLLMHttpServer pid=2050805) WARNING 07-02 07:33:26 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned
(WorkerDict pid=2049936) WARNING 07-02 07:33:34 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'.
(vLLMHttpServer pid=2050803) WARNING 07-02 07:33:27 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster]
(WorkerDict pid=2049934) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2049934) 2026-07-02 07:33:49,587 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ...
(WorkerDict pid=2049934) 2026-07-02 07:33:49,606 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends
(vLLMHttpServer pid=2050803) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster]
(vLLMHttpServer pid=2050803) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster]
(vLLMHttpServer pid=2050803) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster]
(vLLMHttpServer pid=2050803) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 4%|▍ | 2/51 [00:00<00:02, 18.95it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 8%|▊ | 4/51 [00:00<00:02, 19.08it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 12%|█▏ | 6/51 [00:00<00:02, 18.34it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 16%|█▌ | 8/51 [00:00<00:02, 18.53it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 20%|█▉ | 10/51 [00:00<00:02, 18.28it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 24%|██▎ | 12/51 [00:00<00:02, 18.04it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 27%|██▋ | 14/51 [00:00<00:02, 17.66it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 31%|███▏ | 16/51 [00:00<00:02, 17.34it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 35%|███▌ | 18/51 [00:01<00:01, 16.84it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 39%|███▉ | 20/51 [00:01<00:01, 16.81it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 43%|████▎ | 22/51 [00:01<00:01, 16.69it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 47%|████▋ | 24/51 [00:01<00:01, 16.50it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 51%|█████ | 26/51 [00:01<00:01, 16.35it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 55%|█████▍ | 28/51 [00:01<00:01, 16.28it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 59%|█████▉ | 30/51 [00:01<00:01, 16.24it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 63%|██████▎ | 32/51 [00:01<00:01, 16.10it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 67%|██████▋ | 34/51 [00:02<00:01, 15.64it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 71%|███████ | 36/51 [00:02<00:00, 15.37it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 75%|███████▍ | 38/51 [00:02<00:00, 15.19it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 78%|███████▊ | 40/51 [00:02<00:00, 14.63it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 82%|████████▏ | 42/51 [00:02<00:00, 14.75it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 86%|████████▋ | 44/51 [00:02<00:00, 14.72it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 90%|█████████ | 46/51 [00:02<00:00, 14.63it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 94%|█████████▍| 48/51 [00:02<00:00, 14.68it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 98%|█████████▊| 50/51 [00:03<00:00, 15.05it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 16.00it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 0%| | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 6%|▌ | 3/51 [00:00<00:02, 19.79it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 10%|▉ | 5/51 [00:00<00:02, 19.78it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 16%|█▌ | 8/51 [00:00<00:02, 21.09it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 22%|██▏ | 11/51 [00:00<00:01, 21.75it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 27%|██▋ | 14/51 [00:00<00:01, 20.35it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 33%|███▎ | 17/51 [00:00<00:01, 21.07it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 39%|███▉ | 20/51 [00:00<00:01, 21.63it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 45%|████▌ | 23/51 [00:01<00:01, 21.93it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 51%|█████ | 26/51 [00:01<00:01, 22.07it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 57%|█████▋ | 29/51 [00:01<00:00, 22.19it/s]
(WorkerDict pid=2049938) 2026-07-02 07:33:49,673 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... [repeated 7x across cluster]
(WorkerDict pid=2049938) 2026-07-02 07:33:49,712 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends [repeated 7x across cluster]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 63%|██████▎ | 32/51 [00:01<00:00, 22.40it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 69%|██████▊ | 35/51 [00:01<00:00, 22.63it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 75%|███████▍ | 38/51 [00:01<00:00, 22.87it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 80%|████████ | 41/51 [00:01<00:00, 22.82it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 86%|████████▋ | 44/51 [00:01<00:00, 22.86it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 92%|█████████▏| 47/51 [00:02<00:00, 22.90it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 98%|█████████▊| 50/51 [00:02<00:00, 23.00it/s]
(WorkerDict pid=2049932) Capturing CUDA graphs (decode, FULL): 100%|██████████| 51/51 [00:02<00:00, 22.16it/s]
(vLLMHttpServer pid=2050804) WARNING 07-02 07:33:57 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development!
(WorkerDict pid=2049939) WARNING 07-02 07:33:35 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. [repeated 7x across cluster]
(WorkerDict pid=2049938) NCCL version 2.27.5+cuda12.9 [repeated 2x across cluster]
(vLLMHttpServer pid=2050804) WARNING 07-02 07:33:58 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(TaskRunner pid=2049544) AgentLoopManager: ['198.19.44.212:34195', '198.19.44.212:39727', '198.19.44.212:41109', '198.19.44.212:36915']
(TaskRunner pid=2049544) wandb: Currently logged in as: seongryongjung (seongryongjung-chung-ang-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
(TaskRunner pid=2049544) wandb: setting up run o8ivivjg
(TaskRunner pid=2049544) wandb: Tracking run with wandb version 0.23.1
(TaskRunner pid=2049544) wandb: Run data is saved locally in /mnt/mole/SDPO/L2T/wandb/run-20260702_073403-o8ivivjg
(TaskRunner pid=2049544) wandb: Run `wandb offline` to turn off syncing.
(TaskRunner pid=2049544) wandb: Syncing run qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8
(TaskRunner pid=2049544) wandb: ⭐️ View project at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root
(TaskRunner pid=2049544) wandb: 🚀 View run at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/o8ivivjg
(TaskRunner pid=2049544) Checkpoint tracker file does not exist: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/latest_checkpointed_iteration.txt
(TaskRunner pid=2049544) Training from scratch
(vLLMHttpServer pid=2050806) WARNING 07-02 07:33:58 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development! [repeated 3x across cluster]
(vLLMHttpServer pid=2050806) WARNING 07-02 07:33:58 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 3x across cluster]
(TaskRunner pid=2049544) wandb: Detected [openai] in use.
(TaskRunner pid=2049544) wandb: Use W&B Weave for improved LLM call tracing. Install Weave with `pip install weave` then add `import weave` to the top of your script.
(TaskRunner pid=2049544) wandb: For more information, check out the docs at: https://weave-docs.wandb.ai/
(TaskRunner pid=2049544) Training Progress: 0%| | 0/100 [00:00<?, ?it/s]
(AgentLoopWorker pid=2051766) Using dataset class: RLHFDataset
(AgentLoopWorker pid=2051766) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(AgentLoopWorker pid=2051766) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(AgentLoopWorker pid=2051765) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
(AgentLoopWorker pid=2051764) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(AgentLoopWorker pid=2051764) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(TaskRunner pid=2049544) step:1 - global_seqlen/min:18565 - global_seqlen/max:27597 - global_seqlen/minmax_diff:9032 - global_seqlen/balanced_min:22760 - global_seqlen/balanced_max:22772 - global_seqlen/mean:22767.5 - actor/entropy:0.12519954144954681 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45349442958831787 - training/rollout_probs_diff_mean:0.003629719838500023 - training/rollout_probs_diff_std:0.013431069441139698 - training/rollout_actor_probs_pearson_corr:0.9972897171974182 - rollout_corr/rollout_is_mean:1.000240445137024 - rollout_corr/rollout_is_ratio_fraction_high:5.4052106861490756e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012612158025149256 - rollout_corr/rollout_is_max:3.694617986679077 - rollout_corr/rollout_is_min:0.12561839818954468 - rollout_corr/rollout_is_std:0.03757545351982117 - rollout_corr/rollout_is_eff_sample_size:0.9985905513610702 - rollout_corr/rollout_is_seq_mean:1.0000557899475098 - rollout_corr/rollout_is_seq_std:0.00250430079177022 - rollout_corr/rollout_is_seq_max:1.0093392133712769 - rollout_corr/rollout_is_seq_min:0.9886724352836609 - rollout_corr/rollout_is_seq_max_deviation:0.011327564716339111 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1275503737851977) - rollout_corr/training_log_ppl:np.float64(0.11614006559830159) - rollout_corr/kl:np.float64(0.0007402765363817565) - rollout_corr/k3_kl:np.float64(0.0007960655748888712) - rollout_corr/rollout_ppl:np.float64(1.1267359307967126) - rollout_corr/rollout_log_ppl:np.float64(0.11539978917426197) - rollout_corr/log_ppl_diff:np.float64(0.0007402764240396209) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018534858027123846) - rollout_corr/log_ppl_diff_max:np.float64(0.01321200281381607) - rollout_corr/log_ppl_diff_min:np.float64(-0.007948264479637146) - rollout_corr/ppl_ratio:np.float64(1.0007436126470566) - rollout_corr/chi2_token:np.float64(0.001790371723473072) - rollout_corr/chi2_seq:np.float64(2.2340067769400775) - actor/pg_loss:np.float64(9.620818309485912e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.49060917645692825) - perf/mfu/actor:np.float64(0.06213226611896161) - perf/max_memory_allocated_gb:np.float64(116.22064924240112) - perf/max_memory_reserved_gb:np.float64(120.02734375) - perf/cpu_memory_used_gb:np.float64(98.49499130249023) - actor/lr:np.float64(0.0) - training/global_step:1 - training/epoch:0 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002052854048088193 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002052854048088193 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:433.609375 - response_length/max:1342.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:433.609375 - response_length_non_aborted/max:1342.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.875 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002136286348104477 - timing_s/agent_loop/generate_sequences/min:np.float64(8.413902202621102) - timing_s/agent_loop/generate_sequences/max:np.float64(23.541557354852557) - timing_s/agent_loop/generate_sequences/mean:np.float64(16.40960601693223) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.541557354852557) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) -
(AgentLoopWorker pid=2051770) Using dataset class: RLHFDataset [repeated 7x across cluster]
(TaskRunner pid=2049544) Training Progress: 1%| | 1/100 [00:49<1:22:17, 49.87s/it]
(AgentLoopWorker pid=2051768) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding. [repeated 7x across cluster]
(TaskRunner pid=2049544) step:2 - global_seqlen/min:17909 - global_seqlen/max:29315 - global_seqlen/minmax_diff:11406 - global_seqlen/balanced_min:22880 - global_seqlen/balanced_max:22891 - global_seqlen/mean:22884.0 - actor/entropy:0.11217048764228821 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9999963641166687 - training/rollout_probs_diff_mean:0.003297744784504175 - training/rollout_probs_diff_std:0.013333442620933056 - training/rollout_actor_probs_pearson_corr:0.9970114827156067 - rollout_corr/rollout_is_mean:1.000024437904358 - rollout_corr/rollout_is_ratio_fraction_high:5.239991514827125e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011353315494488925 - rollout_corr/rollout_is_max:6.762055397033691 - rollout_corr/rollout_is_min:6.475970621977467e-07 - rollout_corr/rollout_is_std:0.03503736853599548 - rollout_corr/rollout_is_eff_sample_size:0.9987736500983223 - rollout_corr/rollout_is_seq_mean:1.000178575515747 - rollout_corr/rollout_is_seq_std:0.00298739830031991 - rollout_corr/rollout_is_seq_max:1.0262460708618164 - rollout_corr/rollout_is_seq_min:0.9870814681053162 - rollout_corr/rollout_is_seq_max_deviation:0.026246070861816406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1167702935636044) - rollout_corr/training_log_ppl:np.float64(0.10843055222721887) - rollout_corr/kl:np.float64(0.0007375286082371701) - rollout_corr/k3_kl:np.float64(0.0010175542608124033) - rollout_corr/rollout_ppl:np.float64(1.1159212864004076) - rollout_corr/rollout_log_ppl:np.float64(0.10769302266999148) - rollout_corr/log_ppl_diff:np.float64(0.0007375295572273899) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017680755954643246) - rollout_corr/log_ppl_diff_max:np.float64(0.03630320727825165) - rollout_corr/log_ppl_diff_min:np.float64(-0.009953729808330536) - rollout_corr/ppl_ratio:np.float64(1.0007433104328811) - rollout_corr/chi2_token:np.float64(0.0031598308123648167) - rollout_corr/chi2_seq:np.float64(0.8496042392216623) - actor/pg_loss:np.float64(0.00014715001452714205) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000367845517303067) - actor/ppo_kl:np.float64(-5.3390779659379106e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.287675466388464) - perf/mfu/actor:np.float64(0.06504165169288892) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.37890625) - perf/cpu_memory_used_gb:np.float64(99.20343780517578) - actor/lr:np.float64(1e-07) - training/global_step:2 - training/epoch:0 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004377576522529125 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004377576522529125 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:447.28125 - response_length/max:1641.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:447.28125 - response_length_non_aborted/max:1641.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.84375 - prompt_length/max:342.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001490507274866104 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8143141772598028) - timing_s/agent_loop/generate_sequences/max:np.float64(10.941161828115582) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.471072738138901) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.941161828115582) - timing_s/agent_loop/s
(TaskRunner pid=2049544) Training Progress: 2%|▏ | 2/100 [01:20<1:02:44, 38.41s/it]
(TaskRunner pid=2049544) step:3 - global_seqlen/min:16890 - global_seqlen/max:29519 - global_seqlen/minmax_diff:12629 - global_seqlen/balanced_min:23206 - global_seqlen/balanced_max:23209 - global_seqlen/mean:23207.875 - actor/entropy:0.13150057196617126 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35351744294166565 - training/rollout_probs_diff_mean:0.003573887748643756 - training/rollout_probs_diff_std:0.01286943256855011 - training/rollout_actor_probs_pearson_corr:0.9974791407585144 - rollout_corr/rollout_is_mean:1.000145435333252 - rollout_corr/rollout_is_ratio_fraction_high:3.448543429840356e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.310679287300445e-05 - rollout_corr/rollout_is_max:2.740840196609497 - rollout_corr/rollout_is_min:0.3028872609138489 - rollout_corr/rollout_is_std:0.03569132089614868 - rollout_corr/rollout_is_eff_sample_size:0.9987281069318827 - rollout_corr/rollout_is_seq_mean:1.0002286434173584 - rollout_corr/rollout_is_seq_std:0.0023261811584234238 - rollout_corr/rollout_is_seq_max:1.0112227201461792 - rollout_corr/rollout_is_seq_min:0.9917528629302979 - rollout_corr/rollout_is_seq_max_deviation:0.0112227201461792 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1272960603237152) - rollout_corr/training_log_ppl:np.float64(0.11730807536878274) - rollout_corr/kl:np.float64(0.00045615166413881525) - rollout_corr/k3_kl:np.float64(0.0006645458873890675) - rollout_corr/rollout_ppl:np.float64(1.1267952509224415) - rollout_corr/rollout_log_ppl:np.float64(0.11685192224103957) - rollout_corr/log_ppl_diff:np.float64(0.0004561531277431641) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017133288019977044) - rollout_corr/log_ppl_diff_max:np.float64(0.009288642555475235) - rollout_corr/log_ppl_diff_min:np.float64(-0.009320229291915894) - rollout_corr/ppl_ratio:np.float64(1.0004588288720697) - rollout_corr/chi2_token:np.float64(0.0017856701742857695) - rollout_corr/chi2_seq:np.float64(1.4539839446078986) - actor/pg_loss:np.float64(-0.00013296271208673716) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00046258484894678986) - actor/ppo_kl:np.float64(1.2125353557657093e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4521970599889755) - perf/mfu/actor:np.float64(0.06752190573012075) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.37890625) - perf/cpu_memory_used_gb:np.float64(99.25334596633911) - actor/lr:np.float64(2e-07) - training/global_step:3 - training/epoch:0 - critic/score/mean:0.5546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007829271256923676 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007829271256923676 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.08984375 - response_length/max:1215.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.08984375 - response_length_non_aborted/max:1215.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.15625 - prompt_length/max:365.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014126114547252655 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9209608379751444) - timing_s/agent_loop/generate_sequences/max:np.float64(9.210840035229921) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.527679403181537) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.210840035229921) - timing_s/a
(TaskRunner pid=2049544) Training Progress: 3%|▎ | 3/100 [01:48<54:38, 33.79s/it]
(TaskRunner pid=2049544) step:4 - global_seqlen/min:16367 - global_seqlen/max:30221 - global_seqlen/minmax_diff:13854 - global_seqlen/balanced_min:22366 - global_seqlen/balanced_max:22387 - global_seqlen/mean:22380.375 - actor/entropy:0.11857572197914124 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3990696668624878 - training/rollout_probs_diff_mean:0.003437632694840431 - training/rollout_probs_diff_std:0.01286217663437128 - training/rollout_actor_probs_pearson_corr:0.9973838925361633 - rollout_corr/rollout_is_mean:1.0000953674316406 - rollout_corr/rollout_is_ratio_fraction_high:3.626769102993421e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001541376841487363 - rollout_corr/rollout_is_max:3.2698984146118164 - rollout_corr/rollout_is_min:0.32924649119377136 - rollout_corr/rollout_is_std:0.03624645620584488 - rollout_corr/rollout_is_eff_sample_size:0.9986882749303362 - rollout_corr/rollout_is_seq_mean:1.000162959098816 - rollout_corr/rollout_is_seq_std:0.002771026687696576 - rollout_corr/rollout_is_seq_max:1.0198938846588135 - rollout_corr/rollout_is_seq_min:0.9917742609977722 - rollout_corr/rollout_is_seq_max_deviation:0.019893884658813477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.119651960209012) - rollout_corr/training_log_ppl:np.float64(0.11032137211441295) - rollout_corr/kl:np.float64(0.0004552794046652764) - rollout_corr/k3_kl:np.float64(0.0007244579003504725) - rollout_corr/rollout_ppl:np.float64(1.1191515820100904) - rollout_corr/rollout_log_ppl:np.float64(0.10986609250539914) - rollout_corr/log_ppl_diff:np.float64(0.0004552796090138145) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016851588225108571) - rollout_corr/log_ppl_diff_max:np.float64(0.010989315807819366) - rollout_corr/log_ppl_diff_min:np.float64(-0.010053463280200958) - rollout_corr/ppl_ratio:np.float64(1.0004581767134368) - rollout_corr/chi2_token:np.float64(0.002104576211422682) - rollout_corr/chi2_seq:np.float64(2.530068305786699) - actor/pg_loss:np.float64(0.00011606293264776468) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005593107275672082) - actor/ppo_kl:np.float64(-0.00013355799111991473) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4614599272608757) - perf/mfu/actor:np.float64(0.06504037811651654) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.37890625) - perf/cpu_memory_used_gb:np.float64(99.26691627502441) - actor/lr:np.float64(3e-07) - training/global_step:4 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01671147160232067 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01671147160232067 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:430.82421875 - response_length/max:1259.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:430.82421875 - response_length_non_aborted/max:1259.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:361.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010091438889503479 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0971320625394583) - timing_s/agent_loop/generate_sequences/max:np.float64(9.310389595106244) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.469947210949613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.310389595106244) - timing_s/agent_loop/sl
(TaskRunner pid=2049544) Training Progress: 4%|▍ | 4/100 [02:16<50:27, 31.53s/it]
(TaskRunner pid=2049544) step:5 - global_seqlen/min:18495 - global_seqlen/max:25947 - global_seqlen/minmax_diff:7452 - global_seqlen/balanced_min:22554 - global_seqlen/balanced_max:22585 - global_seqlen/mean:22578.5 - actor/entropy:0.10194020718336105 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424376845359802 - training/rollout_probs_diff_mean:0.0029537829104810953 - training/rollout_probs_diff_std:0.011885174550116062 - training/rollout_actor_probs_pearson_corr:0.9973373413085938 - rollout_corr/rollout_is_mean:0.999976396560669 - rollout_corr/rollout_is_ratio_fraction_high:5.20417706866283e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012143079948145896 - rollout_corr/rollout_is_max:3.268083095550537 - rollout_corr/rollout_is_min:0.1355857104063034 - rollout_corr/rollout_is_std:0.03236682340502739 - rollout_corr/rollout_is_eff_sample_size:0.9989533661442945 - rollout_corr/rollout_is_seq_mean:0.9999976754188538 - rollout_corr/rollout_is_seq_std:0.0022474576253443956 - rollout_corr/rollout_is_seq_max:1.0124729871749878 - rollout_corr/rollout_is_seq_min:0.991777241230011 - rollout_corr/rollout_is_seq_max_deviation:0.012472987174987793 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.099419945385307) - rollout_corr/training_log_ppl:np.float64(0.09259689100144897) - rollout_corr/kl:np.float64(0.0005924740671314765) - rollout_corr/k3_kl:np.float64(0.0006842085819158683) - rollout_corr/rollout_ppl:np.float64(1.0987579617649317) - rollout_corr/rollout_log_ppl:np.float64(0.0920044171743939) - rollout_corr/log_ppl_diff:np.float64(0.0005924738270550733) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016580242136114975) - rollout_corr/log_ppl_diff_max:np.float64(0.011860966682434082) - rollout_corr/log_ppl_diff_min:np.float64(-0.006724588572978973) - rollout_corr/ppl_ratio:np.float64(1.0005951295606792) - rollout_corr/chi2_token:np.float64(0.0016028012614697218) - rollout_corr/chi2_seq:np.float64(2.4051406448706985) - actor/pg_loss:np.float64(7.789558731019497e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003204041886419873) - actor/ppo_kl:np.float64(-6.613892763596141e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20145257748663425) - perf/mfu/actor:np.float64(0.06454313942119547) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.37890625) - perf/cpu_memory_used_gb:np.float64(99.09682083129883) - actor/lr:np.float64(4e-07) - training/global_step:5 - training/epoch:0 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004336814396083355 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004336814396083355 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:450.359375 - response_length/max:1689.0 - response_length/min:68.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:450.359375 - response_length_non_aborted/max:1689.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.21875 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012886524200439453 - timing_s/agent_loop/generate_sequences/min:np.float64(0.5393032822757959) - timing_s/agent_loop/generate_sequences/max:np.float64(11.18605532310903) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.282261268803268) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.18605532310903) - timing_s/agent_loop/
(TaskRunner pid=2049544) Training Progress: 5%|▌ | 5/100 [02:46<49:05, 31.01s/it]
(TaskRunner pid=2049544) step:6 - global_seqlen/min:18247 - global_seqlen/max:32702 - global_seqlen/minmax_diff:14455 - global_seqlen/balanced_min:24517 - global_seqlen/balanced_max:24529 - global_seqlen/mean:24525.75 - actor/entropy:0.12611687183380127 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7017514705657959 - training/rollout_probs_diff_mean:0.0036589219234883785 - training/rollout_probs_diff_std:0.013624191284179688 - training/rollout_actor_probs_pearson_corr:0.9972048997879028 - rollout_corr/rollout_is_mean:1.000200629234314 - rollout_corr/rollout_is_ratio_fraction_high:2.4412869606749155e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020344059157650918 - rollout_corr/rollout_is_max:11.85377025604248 - rollout_corr/rollout_is_min:0.2524667978286743 - rollout_corr/rollout_is_std:0.03726964816451073 - rollout_corr/rollout_is_eff_sample_size:0.9986134944712988 - rollout_corr/rollout_is_seq_mean:1.000136375427246 - rollout_corr/rollout_is_seq_std:0.002656314754858613 - rollout_corr/rollout_is_seq_max:1.016276478767395 - rollout_corr/rollout_is_seq_min:0.9897667169570923 - rollout_corr/rollout_is_seq_max_deviation:0.01627647876739502 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1345465863123536) - rollout_corr/training_log_ppl:np.float64(0.12387367367045954) - rollout_corr/kl:np.float64(0.0004401803398756954) - rollout_corr/k3_kl:np.float64(0.000931377761688168) - rollout_corr/rollout_ppl:np.float64(1.1340364199131727) - rollout_corr/rollout_log_ppl:np.float64(0.1234334916662192) - rollout_corr/log_ppl_diff:np.float64(0.0004401820042403415) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016901790804695338) - rollout_corr/log_ppl_diff_max:np.float64(0.009920522570610046) - rollout_corr/log_ppl_diff_min:np.float64(-0.007739752531051636) - rollout_corr/ppl_ratio:np.float64(1.000443163793534) - rollout_corr/chi2_token:np.float64(0.0048747798427939415) - rollout_corr/chi2_seq:np.float64(3.999080861452967) - actor/pg_loss:np.float64(0.00037742021959275007) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005150392039467988) - actor/ppo_kl:np.float64(-0.00033583435445283527) - actor/pg_clipfrac_lower:np.float64(1.6075102394097485e-05) - actor/grad_norm:np.float64(0.3401915058493614) - perf/mfu/actor:np.float64(0.06994923709302454) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.37890625) - perf/cpu_memory_used_gb:np.float64(99.09236907958984) - actor/lr:np.float64(5e-07) - training/global_step:6 - training/epoch:0 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006398206460289657 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.0006398206460289657 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:480.0234375 - response_length/max:1252.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.0234375 - response_length_non_aborted/max:1252.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.40625 - prompt_length/max:375.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015555508434772491 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2495550476014614) - timing_s/agent_loop/generate_sequences/max:np.float64(9.809784756973386) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.023544931398646) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.809784756973
(TaskRunner pid=2049544) Training Progress: 6%|▌ | 6/100 [03:15<47:22, 30.24s/it]
(TaskRunner pid=2049544) step:7 - global_seqlen/min:16871 - global_seqlen/max:29873 - global_seqlen/minmax_diff:13002 - global_seqlen/balanced_min:22253 - global_seqlen/balanced_max:22293 - global_seqlen/mean:22269.0 - actor/entropy:0.0966571494936943 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9852267503738403 - training/rollout_probs_diff_mean:0.003142770379781723 - training/rollout_probs_diff_std:0.013071652501821518 - training/rollout_actor_probs_pearson_corr:0.9967014789581299 - rollout_corr/rollout_is_mean:0.9999518990516663 - rollout_corr/rollout_is_ratio_fraction_high:6.425791070796549e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001193361240439117 - rollout_corr/rollout_is_max:3.388598918914795 - rollout_corr/rollout_is_min:0.0007197936065495014 - rollout_corr/rollout_is_std:0.03546011075377464 - rollout_corr/rollout_is_eff_sample_size:0.9987439217121525 - rollout_corr/rollout_is_seq_mean:1.0000123977661133 - rollout_corr/rollout_is_seq_std:0.0025191924069076777 - rollout_corr/rollout_is_seq_max:1.0151814222335815 - rollout_corr/rollout_is_seq_min:0.9881729483604431 - rollout_corr/rollout_is_seq_max_deviation:0.015181422233581543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1013834550976753) - rollout_corr/training_log_ppl:np.float64(0.09501208057190524) - rollout_corr/kl:np.float64(0.0008676601419210783) - rollout_corr/k3_kl:np.float64(0.0008304009908215448) - rollout_corr/rollout_ppl:np.float64(1.100421885959804) - rollout_corr/rollout_log_ppl:np.float64(0.09414441945409635) - rollout_corr/log_ppl_diff:np.float64(0.0008676611178088933) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018786626460496336) - rollout_corr/log_ppl_diff_max:np.float64(0.015846513211727142) - rollout_corr/log_ppl_diff_min:np.float64(-0.008273690938949585) - rollout_corr/ppl_ratio:np.float64(1.0008711912669241) - rollout_corr/chi2_token:np.float64(0.0015676040202379227) - rollout_corr/chi2_seq:np.float64(0.5792204041499645) - actor/pg_loss:np.float64(8.894084021449089e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006613922037104203) - actor/ppo_kl:np.float64(6.039486299158625e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3810753785073757) - perf/mfu/actor:np.float64(0.06393896301748767) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.6328125) - perf/cpu_memory_used_gb:np.float64(99.01763153076172) - actor/lr:np.float64(6e-07) - training/global_step:7 - training/epoch:0 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001715456834062934 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001715456834062934 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:425.53125 - response_length/max:1850.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:425.53125 - response_length_non_aborted/max:1850.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.375 - prompt_length/max:363.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012745335698127747 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8776181917637587) - timing_s/agent_loop/generate_sequences/max:np.float64(11.998973051086068) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.366835491418897) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.998973051086068) - timing_s/agent_loop/
(TaskRunner pid=2049544) Training Progress: 7%|▋ | 7/100 [03:46<47:17, 30.51s/it]
(TaskRunner pid=2049544) step:8 - global_seqlen/min:21860 - global_seqlen/max:31590 - global_seqlen/minmax_diff:9730 - global_seqlen/balanced_min:24576 - global_seqlen/balanced_max:24583 - global_seqlen/mean:24581.375 - actor/entropy:0.11530227959156036 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6848812699317932 - training/rollout_probs_diff_mean:0.0031557991169393063 - training/rollout_probs_diff_std:0.012370445765554905 - training/rollout_actor_probs_pearson_corr:0.9974690675735474 - rollout_corr/rollout_is_mean:0.9999414086341858 - rollout_corr/rollout_is_ratio_fraction_high:4.7198383981594816e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.866397208999842e-05 - rollout_corr/rollout_is_max:3.6763553619384766 - rollout_corr/rollout_is_min:0.13984894752502441 - rollout_corr/rollout_is_std:0.03325514495372772 - rollout_corr/rollout_is_eff_sample_size:0.9988950792026613 - rollout_corr/rollout_is_seq_mean:0.9999129772186279 - rollout_corr/rollout_is_seq_std:0.0018590353429317474 - rollout_corr/rollout_is_seq_max:1.0087512731552124 - rollout_corr/rollout_is_seq_min:0.9944441318511963 - rollout_corr/rollout_is_seq_max_deviation:0.008751273155212402 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1261226064525545) - rollout_corr/training_log_ppl:np.float64(0.11559916392434388) - rollout_corr/kl:np.float64(0.0008338122016251504) - rollout_corr/k3_kl:np.float64(0.0006376900025359955) - rollout_corr/rollout_ppl:np.float64(1.1251448295079172) - rollout_corr/rollout_log_ppl:np.float64(0.11476535141628119) - rollout_corr/log_ppl_diff:np.float64(0.0008338125080626924) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015558924533252139) - rollout_corr/log_ppl_diff_max:np.float64(0.0066384077072143555) - rollout_corr/log_ppl_diff_min:np.float64(-0.006775319576263428) - rollout_corr/ppl_ratio:np.float64(1.0008359323255718) - rollout_corr/chi2_token:np.float64(0.0008956387173384428) - rollout_corr/chi2_seq:np.float64(0.6552089147735387) - actor/pg_loss:np.float64(0.00033438263926655054) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009480541943958087) - actor/ppo_kl:np.float64(0.00012369491518171571) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.5197392404079437) - perf/mfu/actor:np.float64(0.07099693772855992) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.6328125) - perf/cpu_memory_used_gb:np.float64(98.96152877807617) - actor/lr:np.float64(7e-07) - training/global_step:8 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002576244994997978 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002576244994997978 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:496.57421875 - response_length/max:1213.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:496.57421875 - response_length_non_aborted/max:1213.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.59375 - prompt_length/max:375.0 - prompt_length/min:205.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.276104927062988e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.0848392713814974) - timing_s/agent_loop/generate_sequences/max:np.float64(9.747280441224575) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.214426148610073) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.747280441224575) - timing
(TaskRunner pid=2049544) Training Progress: 8%|▊ | 8/100 [04:14<45:45, 29.85s/it]
(TaskRunner pid=2049544) step:9 - global_seqlen/min:17990 - global_seqlen/max:31669 - global_seqlen/minmax_diff:13679 - global_seqlen/balanced_min:22317 - global_seqlen/balanced_max:22328 - global_seqlen/mean:22322.5 - actor/entropy:0.10605289787054062 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.717214822769165 - training/rollout_probs_diff_mean:0.0033211412373930216 - training/rollout_probs_diff_std:0.01307153794914484 - training/rollout_actor_probs_pearson_corr:0.9970008730888367 - rollout_corr/rollout_is_mean:0.9998250603675842 - rollout_corr/rollout_is_ratio_fraction_high:1.8036867913906462e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00015331337635871023 - rollout_corr/rollout_is_max:4.220541000366211 - rollout_corr/rollout_is_min:0.07421503216028214 - rollout_corr/rollout_is_std:0.03529922291636467 - rollout_corr/rollout_is_eff_sample_size:0.9987552182904212 - rollout_corr/rollout_is_seq_mean:0.9996947050094604 - rollout_corr/rollout_is_seq_std:0.002244219183921814 - rollout_corr/rollout_is_seq_max:1.0129303932189941 - rollout_corr/rollout_is_seq_min:0.9896467924118042 - rollout_corr/rollout_is_seq_max_deviation:0.01293039321899414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1193460430949926) - rollout_corr/training_log_ppl:np.float64(0.10960386582883075) - rollout_corr/kl:np.float64(0.001003489435669025) - rollout_corr/k3_kl:np.float64(0.0007836970782051367) - rollout_corr/rollout_ppl:np.float64(1.1181986778974533) - rollout_corr/rollout_log_ppl:np.float64(0.10860037680686219) - rollout_corr/log_ppl_diff:np.float64(0.0010034890219685622) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018514708135626279) - rollout_corr/log_ppl_diff_max:np.float64(0.012498214840888977) - rollout_corr/log_ppl_diff_min:np.float64(-0.005196064710617065) - rollout_corr/ppl_ratio:np.float64(1.0010067382827401) - rollout_corr/chi2_token:np.float64(0.0011986666359007359) - rollout_corr/chi2_seq:np.float64(0.7110905137378722) - actor/pg_loss:np.float64(-1.6048550605773926e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006157968007300951) - actor/ppo_kl:np.float64(-6.931799785014903e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.31399234384298325) - perf/mfu/actor:np.float64(0.06542459369382952) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.6328125) - perf/cpu_memory_used_gb:np.float64(98.88327407836914) - actor/lr:np.float64(8e-07) - training/global_step:9 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003199289320036769 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003199289320036769 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:433.140625 - response_length/max:1677.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:433.140625 - response_length_non_aborted/max:1677.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.4375 - prompt_length/max:362.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00021540746092796326 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8261344414204359) - timing_s/agent_loop/generate_sequences/max:np.float64(10.815302653238177) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.318218888416595) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.815302653238177) - timing_s/ag
(TaskRunner pid=2049544) Training Progress: 9%|▉ | 9/100 [04:44<45:07, 29.76s/it]
(TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 10}
(TaskRunner pid=2049544) validation generation end
(TaskRunner pid=2049544) [prompt] system
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2049544) <reasoning>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) A
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) user
(TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) A: 50 pC
(TaskRunner pid=2049544) B: 56 pC
(TaskRunner pid=2049544) C: 60 pC
(TaskRunner pid=2049544) D: 64 pC
(TaskRunner pid=2049544) Please reason step by step.
(TaskRunner pid=2049544) assistant
(TaskRunner pid=2049544) <think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) [response] <reasoning>
(TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) E = k \frac{q}{r^2}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Rearranging to solve for $ q $:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Substituting the given values:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $):
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) The closest option is B: 56 pC.
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) B
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544) [ground_truth] B
(TaskRunner pid=2049544) [score] 1.0
(TaskRunner pid=2049544) [acc] 1.0
(TaskRunner pid=2049544) [pred] B
(TaskRunner pid=2049544) [incorrect_format] 1
(TaskRunner pid=2049544) [feedback]
(TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10
(TaskRunner pid=2049544) step:10 - global_seqlen/min:15442 - global_seqlen/max:25018 - global_seqlen/minmax_diff:9576 - global_seqlen/balanced_min:21213 - global_seqlen/balanced_max:21224 - global_seqlen/mean:21220.5 - actor/entropy:0.12735222280025482 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40465062856674194 - training/rollout_probs_diff_mean:0.0038589579053223133 - training/rollout_probs_diff_std:0.013698565773665905 - training/rollout_actor_probs_pearson_corr:0.9970982670783997 - rollout_corr/rollout_is_mean:1.0000947713851929 - rollout_corr/rollout_is_ratio_fraction_high:3.965264113503508e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020817638142034411 - rollout_corr/rollout_is_max:3.167215347290039 - rollout_corr/rollout_is_min:0.19278468191623688 - rollout_corr/rollout_is_std:0.03876073658466339 - rollout_corr/rollout_is_eff_sample_size:0.9985000968313013 - rollout_corr/rollout_is_seq_mean:1.000048041343689 - rollout_corr/rollout_is_seq_std:0.002764123724773526 - rollout_corr/rollout_is_seq_max:1.0100802183151245 - rollout_corr/rollout_is_seq_min:0.9903356432914734 - rollout_corr/rollout_is_seq_max_deviation:0.010080218315124512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1353368978016078) - rollout_corr/training_log_ppl:np.float64(0.12389787597203394) - rollout_corr/kl:np.float64(0.000877203190441378) - rollout_corr/k3_kl:np.float64(0.0008888360377170557) - rollout_corr/rollout_ppl:np.float64(1.134331472683698) - rollout_corr/rollout_log_ppl:np.float64(0.12302067201380851) - rollout_corr/log_ppl_diff:np.float64(0.0008772039582254365) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020878618815913796) - rollout_corr/log_ppl_diff_max:np.float64(0.012568935751914978) - rollout_corr/log_ppl_diff_min:np.float64(-0.007140792906284332) - rollout_corr/ppl_ratio:np.float64(1.000881330575794) - rollout_corr/chi2_token:np.float64(0.0018501763697713614) - rollout_corr/chi2_seq:np.float64(0.666111497906968) - actor/pg_loss:np.float64(9.953684639185667e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005019576683480409) - actor/ppo_kl:np.float64(2.3842763525294686e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3116539753973484) - perf/mfu/actor:np.float64(0.0619170865443955) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.6328125) - perf/cpu_memory_used_gb:np.float64(98.92141723632812) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1013687608239918) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.777675) - val-aux/sciknoweval/reward/best@1
(TaskRunner pid=2049544) Training Progress: 10%|█ | 10/100 [06:47<1:27:44, 58.49s/it]
(TaskRunner pid=2049544) step:11 - global_seqlen/min:16821 - global_seqlen/max:31114 - global_seqlen/minmax_diff:14293 - global_seqlen/balanced_min:23187 - global_seqlen/balanced_max:29684 - global_seqlen/mean:24008.625 - actor/entropy:0.11019539088010788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948736071586609 - training/rollout_probs_diff_mean:0.003139072796329856 - training/rollout_probs_diff_std:0.012617148458957672 - training/rollout_actor_probs_pearson_corr:0.997302770614624 - rollout_corr/rollout_is_mean:1.0000996589660645 - rollout_corr/rollout_is_ratio_fraction_high:7.389344682451338e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.852459334069863e-05 - rollout_corr/rollout_is_max:4.887231826782227 - rollout_corr/rollout_is_min:0.26104938983917236 - rollout_corr/rollout_is_std:0.03467484563589096 - rollout_corr/rollout_is_eff_sample_size:0.9987993368491189 - rollout_corr/rollout_is_seq_mean:1.0002052783966064 - rollout_corr/rollout_is_seq_std:0.0023445014376193285 - rollout_corr/rollout_is_seq_max:1.0106254816055298 - rollout_corr/rollout_is_seq_min:0.99307781457901 - rollout_corr/rollout_is_seq_max_deviation:0.010625481605529785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1195416245609522) - rollout_corr/training_log_ppl:np.float64(0.1096741780929733) - rollout_corr/kl:np.float64(0.0007273018066071835) - rollout_corr/k3_kl:np.float64(0.0009105393462505162) - rollout_corr/rollout_ppl:np.float64(1.1187340491451323) - rollout_corr/rollout_log_ppl:np.float64(0.10894687551262905) - rollout_corr/log_ppl_diff:np.float64(0.0007273025803442579) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019026359186682384) - rollout_corr/log_ppl_diff_max:np.float64(0.022861547768115997) - rollout_corr/log_ppl_diff_min:np.float64(-0.007030755281448364) - rollout_corr/ppl_ratio:np.float64(1.000731429317966) - rollout_corr/chi2_token:np.float64(0.002295879879966378) - rollout_corr/chi2_seq:np.float64(2.1700741790700704) - actor/pg_loss:np.float64(8.732720743864775e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011307528586712579) - actor/ppo_kl:np.float64(0.00020700877354862257) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.46738822013139725) - perf/mfu/actor:np.float64(0.06503294604815439) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.65916442871094) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01939600333571434 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01939600333571434 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:475.76953125 - response_length/max:8192.0 - response_length/min:69.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:475.76953125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001276358962059021 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8453756030648947) - timing_s/agent_loop/generate_sequences/max:np.float64(51.0958620775491) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.681928445308586) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.0958620775491) - timing_
(TaskRunner pid=2049544) Training Progress: 11%|█ | 11/100 [07:58<1:32:18, 62.23s/it]
(TaskRunner pid=2049544) step:12 - global_seqlen/min:18163 - global_seqlen/max:27095 - global_seqlen/minmax_diff:8932 - global_seqlen/balanced_min:23218 - global_seqlen/balanced_max:23238 - global_seqlen/mean:23234.125 - actor/entropy:0.11550334095954895 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4828122854232788 - training/rollout_probs_diff_mean:0.0033159777522087097 - training/rollout_probs_diff_std:0.012482362799346447 - training/rollout_actor_probs_pearson_corr:0.9973734021186829 - rollout_corr/rollout_is_mean:0.9999693036079407 - rollout_corr/rollout_is_ratio_fraction_high:5.898561721551232e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010111819574376568 - rollout_corr/rollout_is_max:2.5878732204437256 - rollout_corr/rollout_is_min:0.125954732298851 - rollout_corr/rollout_is_std:0.03412209078669548 - rollout_corr/rollout_is_eff_sample_size:0.9988369179946218 - rollout_corr/rollout_is_seq_mean:1.0000054836273193 - rollout_corr/rollout_is_seq_std:0.0017772279679775238 - rollout_corr/rollout_is_seq_max:1.008151888847351 - rollout_corr/rollout_is_seq_min:0.9952659010887146 - rollout_corr/rollout_is_seq_max_deviation:0.008151888847351074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1121750511229038) - rollout_corr/training_log_ppl:np.float64(0.10409450641964213) - rollout_corr/kl:np.float64(0.0006273885898906428) - rollout_corr/k3_kl:np.float64(0.000663624501786586) - rollout_corr/rollout_ppl:np.float64(1.1114575085230172) - rollout_corr/rollout_log_ppl:np.float64(0.10346711749298265) - rollout_corr/log_ppl_diff:np.float64(0.0006273889266594779) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014779563243791927) - rollout_corr/log_ppl_diff_max:np.float64(0.006399750709533691) - rollout_corr/log_ppl_diff_min:np.float64(-0.006532169878482819) - rollout_corr/ppl_ratio:np.float64(1.000629240879789) - rollout_corr/chi2_token:np.float64(0.0014743746723979712) - rollout_corr/chi2_seq:np.float64(1.0935420689638704) - actor/pg_loss:np.float64(0.00015071697998791933) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004757110639275197) - actor/ppo_kl:np.float64(2.2654555305479107e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2736458256840706) - perf/mfu/actor:np.float64(0.0671891188671343) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.26230239868164) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0065000043250620365 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0065000043250620365 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:463.56640625 - response_length/max:1365.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:463.56640625 - response_length_non_aborted/max:1365.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.028241038322449e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0592438988387585) - timing_s/agent_loop/generate_sequences/max:np.float64(10.160819062963128) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.8243368406328955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.160819062963128) - timing_s/agent
(TaskRunner pid=2049544) Training Progress: 12%|█▏ | 12/100 [08:26<1:16:22, 52.08s/it]
(TaskRunner pid=2049544) step:13 - global_seqlen/min:20734 - global_seqlen/max:28493 - global_seqlen/minmax_diff:7759 - global_seqlen/balanced_min:23965 - global_seqlen/balanced_max:23977 - global_seqlen/mean:23970.125 - actor/entropy:0.1322651505470276 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5841533541679382 - training/rollout_probs_diff_mean:0.003578777192160487 - training/rollout_probs_diff_std:0.01281787734478712 - training/rollout_actor_probs_pearson_corr:0.997634768486023 - rollout_corr/rollout_is_mean:0.9998666048049927 - rollout_corr/rollout_is_ratio_fraction_high:8.307234566018451e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00010799404844874516 - rollout_corr/rollout_is_max:2.1130197048187256 - rollout_corr/rollout_is_min:0.289154976606369 - rollout_corr/rollout_is_std:0.03564285859465599 - rollout_corr/rollout_is_eff_sample_size:0.9987307228760026 - rollout_corr/rollout_is_seq_mean:0.9998607635498047 - rollout_corr/rollout_is_seq_std:0.0017514645587652922 - rollout_corr/rollout_is_seq_max:1.0048136711120605 - rollout_corr/rollout_is_seq_min:0.9948503971099854 - rollout_corr/rollout_is_seq_max_deviation:0.0051496028900146484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1441420195624232) - rollout_corr/training_log_ppl:np.float64(0.12973640582640655) - rollout_corr/kl:np.float64(0.0009200233825930049) - rollout_corr/k3_kl:np.float64(0.000771395374357553) - rollout_corr/rollout_ppl:np.float64(1.1430599559098482) - rollout_corr/rollout_log_ppl:np.float64(0.12881638239923632) - rollout_corr/log_ppl_diff:np.float64(0.0009200234271702357) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016153893011505716) - rollout_corr/log_ppl_diff_max:np.float64(0.010122627019882202) - rollout_corr/log_ppl_diff_min:np.float64(-0.007734514772891998) - rollout_corr/ppl_ratio:np.float64(1.0009225921239704) - rollout_corr/chi2_token:np.float64(0.0012212155852466822) - rollout_corr/chi2_seq:np.float64(0.878853014903143) - actor/pg_loss:np.float64(-4.01295255869627e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007891653215210681) - actor/ppo_kl:np.float64(0.00011588802720652325) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.40201497077941895) - perf/mfu/actor:np.float64(0.0691041251464859) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.31333541870117) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005807795561850071 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005807795561850071 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:470.22265625 - response_length/max:1333.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:470.22265625 - response_length_non_aborted/max:1333.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014058127999305725 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4221638720482588) - timing_s/agent_loop/generate_sequences/max:np.float64(9.880332147702575) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.85110625057132) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.880332147702575) - timing_s/ag
(TaskRunner pid=2049544) Training Progress: 13%|█▎ | 13/100 [08:55<1:05:06, 44.90s/it]
(TaskRunner pid=2049544) step:14 - global_seqlen/min:17304 - global_seqlen/max:31074 - global_seqlen/minmax_diff:13770 - global_seqlen/balanced_min:23194 - global_seqlen/balanced_max:23602 - global_seqlen/mean:23258.875 - actor/entropy:0.11846989393234253 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45436424016952515 - training/rollout_probs_diff_mean:0.003607378341257572 - training/rollout_probs_diff_std:0.013258584775030613 - training/rollout_actor_probs_pearson_corr:0.9971495866775513 - rollout_corr/rollout_is_mean:0.9998986721038818 - rollout_corr/rollout_is_ratio_fraction_high:5.1646224164869636e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014633095997851342 - rollout_corr/rollout_is_max:2.4299769401550293 - rollout_corr/rollout_is_min:0.09816896170377731 - rollout_corr/rollout_is_std:0.037202417850494385 - rollout_corr/rollout_is_eff_sample_size:0.9986175363719597 - rollout_corr/rollout_is_seq_mean:0.9998643398284912 - rollout_corr/rollout_is_seq_std:0.002263482892885804 - rollout_corr/rollout_is_seq_max:1.00752854347229 - rollout_corr/rollout_is_seq_min:0.9923214316368103 - rollout_corr/rollout_is_seq_max_deviation:0.007678568363189697 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.13216158375144) - rollout_corr/training_log_ppl:np.float64(0.12105548306863056) - rollout_corr/kl:np.float64(0.0011141863628356674) - rollout_corr/k3_kl:np.float64(0.0007862591612095571) - rollout_corr/rollout_ppl:np.float64(1.1308507332578301) - rollout_corr/rollout_log_ppl:np.float64(0.1199412965834199) - rollout_corr/log_ppl_diff:np.float64(0.0011141864852106664) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018289975014340598) - rollout_corr/log_ppl_diff_max:np.float64(0.01189817488193512) - rollout_corr/log_ppl_diff_min:np.float64(-0.006560362875461578) - rollout_corr/ppl_ratio:np.float64(1.0011174657847732) - rollout_corr/chi2_token:np.float64(0.0008673113770782948) - rollout_corr/chi2_seq:np.float64(0.27274479530751705) - actor/pg_loss:np.float64(8.668552618473768e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010782241606648313) - actor/ppo_kl:np.float64(0.00016768844109904535) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.42187249660491943) - perf/mfu/actor:np.float64(0.06630021670401551) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.41584777832031) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009940821677446365 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009940821677446365 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.80859375 - response_length/max:1963.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.80859375 - response_length_non_aborted/max:1963.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013650394976139069 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0465375203639269) - timing_s/agent_loop/generate_sequences/max:np.float64(12.73308464512229) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.650266676486353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.73308464512229) - timing_
(TaskRunner pid=2049544) Training Progress: 14%|█▍ | 14/100 [09:26<58:38, 40.91s/it]
(TaskRunner pid=2049544) step:15 - global_seqlen/min:15342 - global_seqlen/max:29838 - global_seqlen/minmax_diff:14496 - global_seqlen/balanced_min:23634 - global_seqlen/balanced_max:23672 - global_seqlen/mean:23657.75 - actor/entropy:0.1110260859131813 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9913476705551147 - training/rollout_probs_diff_mean:0.0030704238452017307 - training/rollout_probs_diff_std:0.01288246363401413 - training/rollout_actor_probs_pearson_corr:0.9973083138465881 - rollout_corr/rollout_is_mean:0.9999945163726807 - rollout_corr/rollout_is_ratio_fraction_high:3.303655466879718e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.085052442969754e-05 - rollout_corr/rollout_is_max:3.397390604019165 - rollout_corr/rollout_is_min:0.008578135631978512 - rollout_corr/rollout_is_std:0.03370378911495209 - rollout_corr/rollout_is_eff_sample_size:0.9988653435875123 - rollout_corr/rollout_is_seq_mean:0.9999395608901978 - rollout_corr/rollout_is_seq_std:0.0021285165566951036 - rollout_corr/rollout_is_seq_max:1.016195297241211 - rollout_corr/rollout_is_seq_min:0.9913737177848816 - rollout_corr/rollout_is_seq_max_deviation:0.016195297241210938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.116935707628727) - rollout_corr/training_log_ppl:np.float64(0.10718949149304535) - rollout_corr/kl:np.float64(0.0008361445555777891) - rollout_corr/k3_kl:np.float64(0.0006998892279455049) - rollout_corr/rollout_ppl:np.float64(1.1159698073752224) - rollout_corr/rollout_log_ppl:np.float64(0.10635334601101931) - rollout_corr/log_ppl_diff:np.float64(0.0008361454820260406) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016929644480114803) - rollout_corr/log_ppl_diff_max:np.float64(0.01026906818151474) - rollout_corr/log_ppl_diff_min:np.float64(-0.012994319200515747) - rollout_corr/ppl_ratio:np.float64(1.000839062500745) - rollout_corr/chi2_token:np.float64(0.0011396266054362059) - rollout_corr/chi2_seq:np.float64(1.5898604474496096) - actor/pg_loss:np.float64(-2.7647125534713268e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006954610362299718) - actor/ppo_kl:np.float64(0.00011087026399003719) - actor/pg_clipfrac_lower:np.float64(1.0444518920849077e-05) - actor/grad_norm:np.float64(0.29682732000947) - perf/mfu/actor:np.float64(0.06723820699139785) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.3052020072937) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0049513536505401134 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0049513536505401134 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:472.9609375 - response_length/max:1790.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:472.9609375 - response_length_non_aborted/max:1790.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010203756392002106 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7883450742810965) - timing_s/agent_loop/generate_sequences/max:np.float64(12.01114228926599) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.683357301946671) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.011142289265
(TaskRunner pid=2049544) Training Progress: 15%|█▌ | 15/100 [09:57<53:39, 37.88s/it]
(TaskRunner pid=2049544) step:16 - global_seqlen/min:18400 - global_seqlen/max:27667 - global_seqlen/minmax_diff:9267 - global_seqlen/balanced_min:22148 - global_seqlen/balanced_max:22154 - global_seqlen/mean:22151.125 - actor/entropy:0.12429671734571457 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.739967942237854 - training/rollout_probs_diff_mean:0.0033937261905521154 - training/rollout_probs_diff_std:0.012556870467960835 - training/rollout_actor_probs_pearson_corr:0.9975789189338684 - rollout_corr/rollout_is_mean:1.000154972076416 - rollout_corr/rollout_is_ratio_fraction_high:2.7087302441941574e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.126190368784592e-05 - rollout_corr/rollout_is_max:4.322726726531982 - rollout_corr/rollout_is_min:0.12595464289188385 - rollout_corr/rollout_is_std:0.034952208399772644 - rollout_corr/rollout_is_eff_sample_size:0.9987801905807853 - rollout_corr/rollout_is_seq_mean:1.0001484155654907 - rollout_corr/rollout_is_seq_std:0.0019322986481711268 - rollout_corr/rollout_is_seq_max:1.007652997970581 - rollout_corr/rollout_is_seq_min:0.992692768573761 - rollout_corr/rollout_is_seq_max_deviation:0.007652997970581055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.128569976426661) - rollout_corr/training_log_ppl:np.float64(0.11689590534660965) - rollout_corr/kl:np.float64(0.0006918034323035727) - rollout_corr/k3_kl:np.float64(0.0007205118381676812) - rollout_corr/rollout_ppl:np.float64(1.1277636121958494) - rollout_corr/rollout_log_ppl:np.float64(0.11620410031173378) - rollout_corr/log_ppl_diff:np.float64(0.0006918050348758698) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016668791504343972) - rollout_corr/log_ppl_diff_max:np.float64(0.011440090835094452) - rollout_corr/log_ppl_diff_min:np.float64(-0.005701005458831787) - rollout_corr/ppl_ratio:np.float64(1.000694399466738) - rollout_corr/chi2_token:np.float64(0.0014990693889558315) - rollout_corr/chi2_seq:np.float64(0.6086118693929166) - actor/pg_loss:np.float64(3.552588168531656e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008225217134167906) - actor/ppo_kl:np.float64(0.0001884127358140475) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3684859052300453) - perf/mfu/actor:np.float64(0.05525549139638902) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(118.55257987976074) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014105712994933128 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014105712994933128 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:432.62890625 - response_length/max:1203.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:432.62890625 - response_length_non_aborted/max:1203.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.934604167938232e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0808530896902084) - timing_s/agent_loop/generate_sequences/max:np.float64(8.854041801765561) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.365803626016714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.854041801765561) - timing_s/ag
(TaskRunner pid=2049544) Training Progress: 16%|█▌ | 16/100 [10:26<49:20, 35.25s/it]
(TaskRunner pid=2049544) step:17 - global_seqlen/min:17199 - global_seqlen/max:28844 - global_seqlen/minmax_diff:11645 - global_seqlen/balanced_min:22404 - global_seqlen/balanced_max:28744 - global_seqlen/mean:23204.125 - actor/entropy:0.12878382205963135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7316876649856567 - training/rollout_probs_diff_mean:0.003660086076706648 - training/rollout_probs_diff_std:0.013964627869427204 - training/rollout_actor_probs_pearson_corr:0.9971086978912354 - rollout_corr/rollout_is_mean:0.9999353289604187 - rollout_corr/rollout_is_ratio_fraction_high:5.108600453240797e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.000221372683881782 - rollout_corr/rollout_is_max:3.039975881576538 - rollout_corr/rollout_is_min:0.002080337144434452 - rollout_corr/rollout_is_std:0.037413306534290314 - rollout_corr/rollout_is_eff_sample_size:0.9986019633462979 - rollout_corr/rollout_is_seq_mean:0.9999227523803711 - rollout_corr/rollout_is_seq_std:0.0023962773848325014 - rollout_corr/rollout_is_seq_max:1.0085341930389404 - rollout_corr/rollout_is_seq_min:0.9915060997009277 - rollout_corr/rollout_is_seq_max_deviation:0.00853419303894043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1416860399767756) - rollout_corr/training_log_ppl:np.float64(0.1297323260405392) - rollout_corr/kl:np.float64(0.0012029753426432421) - rollout_corr/k3_kl:np.float64(0.0010000968625991646) - rollout_corr/rollout_ppl:np.float64(1.1402671793475747) - rollout_corr/rollout_log_ppl:np.float64(0.12852934912734781) - rollout_corr/log_ppl_diff:np.float64(0.0012029769131913781) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022540156060131267) - rollout_corr/log_ppl_diff_max:np.float64(0.022689037024974823) - rollout_corr/log_ppl_diff_min:np.float64(-0.0066199153661727905) - rollout_corr/ppl_ratio:np.float64(1.0012089894153178) - rollout_corr/chi2_token:np.float64(0.0014632882084697485) - rollout_corr/chi2_seq:np.float64(0.5507866519037634) - actor/pg_loss:np.float64(0.0002590372459962964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013125582909196964) - actor/ppo_kl:np.float64(0.00025115833264877097) - actor/pg_clipfrac_lower:np.float64(7.70463520893827e-06) - actor/grad_norm:np.float64(0.3952488452196121) - perf/mfu/actor:np.float64(0.06094650606696392) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(110.15214920043945) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022923780605196953 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022923780605196953 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.78515625 - response_length/max:8192.0 - response_length/min:68.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:458.78515625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017055682837963104 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8617520779371262) - timing_s/agent_loop/generate_sequences/max:np.float64(50.32327074185014) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.232911160150252) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.
(TaskRunner pid=2049544) Training Progress: 17%|█▋ | 17/100 [11:38<1:03:45, 46.09s/it]
(TaskRunner pid=2049544) step:18 - global_seqlen/min:17313 - global_seqlen/max:30331 - global_seqlen/minmax_diff:13018 - global_seqlen/balanced_min:25566 - global_seqlen/balanced_max:25574 - global_seqlen/mean:25570.75 - actor/entropy:0.1224943995475769 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5235333442687988 - training/rollout_probs_diff_mean:0.0032062120735645294 - training/rollout_probs_diff_std:0.012459194287657738 - training/rollout_actor_probs_pearson_corr:0.9975793361663818 - rollout_corr/rollout_is_mean:0.9999059438705444 - rollout_corr/rollout_is_ratio_fraction_high:7.298523996723816e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012407491158228368 - rollout_corr/rollout_is_max:2.7950527667999268 - rollout_corr/rollout_is_min:0.06372947990894318 - rollout_corr/rollout_is_std:0.03364891931414604 - rollout_corr/rollout_is_eff_sample_size:0.9988689117678543 - rollout_corr/rollout_is_seq_mean:0.9998814463615417 - rollout_corr/rollout_is_seq_std:0.0018376336665824056 - rollout_corr/rollout_is_seq_max:1.007022738456726 - rollout_corr/rollout_is_seq_min:0.9937419891357422 - rollout_corr/rollout_is_seq_max_deviation:0.007022738456726074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1279504764825106) - rollout_corr/training_log_ppl:np.float64(0.11757075608329615) - rollout_corr/kl:np.float64(0.0008173328948331005) - rollout_corr/k3_kl:np.float64(0.0006817538003787149) - rollout_corr/rollout_ppl:np.float64(1.1270086439326406) - rollout_corr/rollout_log_ppl:np.float64(0.1167534222076938) - rollout_corr/log_ppl_diff:np.float64(0.000817333875602344) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016419640196545515) - rollout_corr/log_ppl_diff_max:np.float64(0.008472427725791931) - rollout_corr/log_ppl_diff_min:np.float64(-0.006251677870750427) - rollout_corr/ppl_ratio:np.float64(1.0008197950664908) - rollout_corr/chi2_token:np.float64(0.0010528604034334421) - rollout_corr/chi2_seq:np.float64(0.8530804882757366) - actor/pg_loss:np.float64(0.0001363652991130948) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009771198135695158) - actor/ppo_kl:np.float64(8.36708865672442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4684727042913437) - perf/mfu/actor:np.float64(0.0710728954199909) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.35284042358398) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004924679175019264 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004924679175019264 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:535.2109375 - response_length/max:1581.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:535.2109375 - response_length_non_aborted/max:1581.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012665800750255585 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8409140948206186) - timing_s/agent_loop/generate_sequences/max:np.float64(11.426093036308885) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7903019126315485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.426093036308885) - timing_s
(TaskRunner pid=2049544) Training Progress: 18%|█▊ | 18/100 [12:08<56:39, 41.46s/it]
(TaskRunner pid=2049544) step:19 - global_seqlen/min:17713 - global_seqlen/max:31095 - global_seqlen/minmax_diff:13382 - global_seqlen/balanced_min:23780 - global_seqlen/balanced_max:23986 - global_seqlen/mean:23818.0 - actor/entropy:0.10652168840169907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43985795974731445 - training/rollout_probs_diff_mean:0.003031388856470585 - training/rollout_probs_diff_std:0.012130273506045341 - training/rollout_actor_probs_pearson_corr:0.9975038766860962 - rollout_corr/rollout_is_mean:1.000024437904358 - rollout_corr/rollout_is_ratio_fraction_high:5.6943903473438695e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010575295891612768 - rollout_corr/rollout_is_max:3.544985771179199 - rollout_corr/rollout_is_min:0.2985078692436218 - rollout_corr/rollout_is_std:0.03386257588863373 - rollout_corr/rollout_is_eff_sample_size:0.9988546391994405 - rollout_corr/rollout_is_seq_mean:1.0000839233398438 - rollout_corr/rollout_is_seq_std:0.002387973479926586 - rollout_corr/rollout_is_seq_max:1.0142525434494019 - rollout_corr/rollout_is_seq_min:0.9915294051170349 - rollout_corr/rollout_is_seq_max_deviation:0.014252543449401855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1093668043613434) - rollout_corr/training_log_ppl:np.float64(0.10132705738578807) - rollout_corr/kl:np.float64(0.0006060766052620181) - rollout_corr/k3_kl:np.float64(0.0007317086579945453) - rollout_corr/rollout_ppl:np.float64(1.1086922818794847) - rollout_corr/rollout_log_ppl:np.float64(0.10072098037198884) - rollout_corr/log_ppl_diff:np.float64(0.0006060770137992222) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016289143422909547) - rollout_corr/log_ppl_diff_max:np.float64(0.014358222484588623) - rollout_corr/log_ppl_diff_min:np.float64(-0.009524352848529816) - rollout_corr/ppl_ratio:np.float64(1.0006090987008065) - rollout_corr/chi2_token:np.float64(0.001838616793975234) - rollout_corr/chi2_seq:np.float64(0.987432076362893) - actor/pg_loss:np.float64(0.0002443152479827404) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007472785102891066) - actor/ppo_kl:np.float64(-8.193314219084868e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4798855558037758) - perf/mfu/actor:np.float64(0.0686439488762442) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.47739028930664) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009253384196199477 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009253384196199477 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:480.1875 - response_length/max:1871.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.1875 - response_length_non_aborted/max:1871.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001460295170545578 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0999032072722912) - timing_s/agent_loop/generate_sequences/max:np.float64(12.776574049144983) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.886515708894876) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.776574049144983) - timing_s/agent_loop/sl
(TaskRunner pid=2049544) Training Progress: 19%|█▉ | 19/100 [12:40<51:52, 38.43s/it]
(TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20}
(TaskRunner pid=2049544) validation generation end
(TaskRunner pid=2049544) [prompt] system
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2049544) <reasoning>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) A
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) user
(TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) A: 50 pC
(TaskRunner pid=2049544) B: 56 pC
(TaskRunner pid=2049544) C: 60 pC
(TaskRunner pid=2049544) D: 64 pC
(TaskRunner pid=2049544) Please reason step by step.
(TaskRunner pid=2049544) assistant
(TaskRunner pid=2049544) <think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) [response] <reasoning>
(TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We are asked to find the charge $ q $ using the formula:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) E = k \frac{q}{r^2}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Rearranging to solve for $ q $:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Substitute the known values:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $):
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) The closest option is **B: 56 pC**.
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) B
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544) [ground_truth] B
(TaskRunner pid=2049544) [score] 1.0
(TaskRunner pid=2049544) [acc] 1.0
(TaskRunner pid=2049544) [pred] B
(TaskRunner pid=2049544) [incorrect_format] 1
(TaskRunner pid=2049544) [feedback]
(TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20
(WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor
(TaskRunner pid=2049544) step:20 - global_seqlen/min:17554 - global_seqlen/max:28214 - global_seqlen/minmax_diff:10660 - global_seqlen/balanced_min:22414 - global_seqlen/balanced_max:22431 - global_seqlen/mean:22418.0 - actor/entropy:0.11887378990650177 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7652756571769714 - training/rollout_probs_diff_mean:0.003253082511946559 - training/rollout_probs_diff_std:0.01256453339010477 - training/rollout_actor_probs_pearson_corr:0.9974572062492371 - rollout_corr/rollout_is_mean:0.9998491406440735 - rollout_corr/rollout_is_ratio_fraction_high:2.658820267242845e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014180374273564667 - rollout_corr/rollout_is_max:2.1670210361480713 - rollout_corr/rollout_is_min:0.20142976939678192 - rollout_corr/rollout_is_std:0.03435710072517395 - rollout_corr/rollout_is_eff_sample_size:0.9988207434840839 - rollout_corr/rollout_is_seq_mean:0.9999091625213623 - rollout_corr/rollout_is_seq_std:0.001986477058380842 - rollout_corr/rollout_is_seq_max:1.0120420455932617 - rollout_corr/rollout_is_seq_min:0.9940481185913086 - rollout_corr/rollout_is_seq_max_deviation:0.012042045593261719 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1164336544461548) - rollout_corr/training_log_ppl:np.float64(0.10743120299957809) - rollout_corr/kl:np.float64(0.0007887022689292422) - rollout_corr/k3_kl:np.float64(0.0007427555091403804) - rollout_corr/rollout_ppl:np.float64(1.1155504160560668) - rollout_corr/rollout_log_ppl:np.float64(0.1066425001481548) - rollout_corr/log_ppl_diff:np.float64(0.0007887028514232952) - rollout_corr/log_ppl_abs_diff:np.float64(0.001653945950238267) - rollout_corr/log_ppl_diff_max:np.float64(0.012162841856479645) - rollout_corr/log_ppl_diff_min:np.float64(-0.010080903768539429) - rollout_corr/ppl_ratio:np.float64(1.0007913769222796) - rollout_corr/chi2_token:np.float64(0.0014412584714591503) - rollout_corr/chi2_seq:np.float64(0.749183313222602) - actor/pg_loss:np.float64(0.00014212285168468952) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000699050789080502) - actor/ppo_kl:np.float64(5.158489911716302e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.37614476680755615) - perf/mfu/actor:np.float64(0.06503999971026932) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.41239166259766) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09981662846229708) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8320624999999999) - val-aux/sciknoweval/reward
(TaskRunner pid=2049544) Training Progress: 20%|██ | 20/100 [14:54<1:29:33, 67.17s/it]
(TaskRunner pid=2049544) step:21 - global_seqlen/min:20890 - global_seqlen/max:29372 - global_seqlen/minmax_diff:8482 - global_seqlen/balanced_min:23784 - global_seqlen/balanced_max:24194 - global_seqlen/mean:23842.25 - actor/entropy:0.10979560017585754 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417163848876953 - training/rollout_probs_diff_mean:0.0029196166433393955 - training/rollout_probs_diff_std:0.01234018336981535 - training/rollout_actor_probs_pearson_corr:0.9974682927131653 - rollout_corr/rollout_is_mean:1.0000228881835938 - rollout_corr/rollout_is_ratio_fraction_high:1.6375992345274426e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010644395661074668 - rollout_corr/rollout_is_max:2.094970703125 - rollout_corr/rollout_is_min:0.07465586066246033 - rollout_corr/rollout_is_std:0.03303755819797516 - rollout_corr/rollout_is_eff_sample_size:0.9989098287246259 - rollout_corr/rollout_is_seq_mean:0.9999141693115234 - rollout_corr/rollout_is_seq_std:0.0017565247835591435 - rollout_corr/rollout_is_seq_max:1.0051144361495972 - rollout_corr/rollout_is_seq_min:0.9947226643562317 - rollout_corr/rollout_is_seq_max_deviation:0.0052773356437683105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1051466539502144) - rollout_corr/training_log_ppl:np.float64(0.09741823082003975) - rollout_corr/kl:np.float64(0.0006470407322218819) - rollout_corr/k3_kl:np.float64(0.0006291852859376945) - rollout_corr/rollout_ppl:np.float64(1.1044287369586527) - rollout_corr/rollout_log_ppl:np.float64(0.09677118871331913) - rollout_corr/log_ppl_diff:np.float64(0.0006470421067206189) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014577525143977255) - rollout_corr/log_ppl_diff_max:np.float64(0.009566932916641235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005102425813674927) - rollout_corr/ppl_ratio:np.float64(1.0006491616368294) - rollout_corr/chi2_token:np.float64(0.0012274319306015968) - rollout_corr/chi2_seq:np.float64(1.2051882504019886) - actor/pg_loss:np.float64(0.00013444910291582346) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045100462170921674) - actor/ppo_kl:np.float64(-5.319785113400144e-05) - actor/pg_clipfrac_lower:np.float64(2.38095403801708e-05) - actor/grad_norm:np.float64(0.2096654698252678) - perf/mfu/actor:np.float64(0.0669203788543201) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.66231536865234) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004104233346879482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004104233346879482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:477.0703125 - response_length/max:2091.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:477.0703125 - response_length_non_aborted/max:2091.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015165284276008606 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7176804710179567) - timing_s/agent_loop/generate_sequences/max:np.float64(12.88374364003539) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.737000476088724) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.883743640035
(TaskRunner pid=2049544) Training Progress: 21%|██ | 21/100 [15:26<1:14:37, 56.68s/it]
(TaskRunner pid=2049544) step:22 - global_seqlen/min:16900 - global_seqlen/max:32996 - global_seqlen/minmax_diff:16096 - global_seqlen/balanced_min:23691 - global_seqlen/balanced_max:23913 - global_seqlen/mean:23732.375 - actor/entropy:0.12188427150249481 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5312178134918213 - training/rollout_probs_diff_mean:0.0033468599431216717 - training/rollout_probs_diff_std:0.013181475922465324 - training/rollout_actor_probs_pearson_corr:0.9972920417785645 - rollout_corr/rollout_is_mean:0.999908983707428 - rollout_corr/rollout_is_ratio_fraction_high:2.4285402105306275e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001942832168424502 - rollout_corr/rollout_is_max:3.412278175354004 - rollout_corr/rollout_is_min:0.14518168568611145 - rollout_corr/rollout_is_std:0.03405214846134186 - rollout_corr/rollout_is_eff_sample_size:0.9988416753032981 - rollout_corr/rollout_is_seq_mean:0.999866247177124 - rollout_corr/rollout_is_seq_std:0.00229041394777596 - rollout_corr/rollout_is_seq_max:1.0077131986618042 - rollout_corr/rollout_is_seq_min:0.9842400550842285 - rollout_corr/rollout_is_seq_max_deviation:0.015759944915771484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1243417421355844) - rollout_corr/training_log_ppl:np.float64(0.11367632360634161) - rollout_corr/kl:np.float64(0.0010524871724442164) - rollout_corr/k3_kl:np.float64(0.000926966261786788) - rollout_corr/rollout_ppl:np.float64(1.1231554276309907) - rollout_corr/rollout_log_ppl:np.float64(0.11262383543362375) - rollout_corr/log_ppl_diff:np.float64(0.001052488172717858) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018730249648797326) - rollout_corr/log_ppl_diff_max:np.float64(0.0278562530875206) - rollout_corr/log_ppl_diff_min:np.float64(-0.004908844828605652) - rollout_corr/ppl_ratio:np.float64(1.0010584215633571) - rollout_corr/chi2_token:np.float64(0.0015142448246479034) - rollout_corr/chi2_seq:np.float64(0.5919819474220276) - actor/pg_loss:np.float64(-3.6185141652822495e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007750144300189277) - actor/ppo_kl:np.float64(0.00017040614556229627) - actor/pg_clipfrac_lower:np.float64(5.271592272038106e-06) - actor/grad_norm:np.float64(0.4317408576607704) - perf/mfu/actor:np.float64(0.06862934905562151) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.66873407363892) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038765573408454657 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0038765573408454657 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:482.54296875 - response_length/max:1955.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:482.54296875 - response_length_non_aborted/max:1955.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013677775859832764 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3632201459258795) - timing_s/agent_loop/generate_sequences/max:np.float64(12.304583990946412) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.897737120372767) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.3045839909464
(TaskRunner pid=2049544) Training Progress: 22%|██▏ | 22/100 [15:58<1:04:00, 49.24s/it]
(TaskRunner pid=2049544) step:23 - global_seqlen/min:17606 - global_seqlen/max:33708 - global_seqlen/minmax_diff:16102 - global_seqlen/balanced_min:26856 - global_seqlen/balanced_max:26886 - global_seqlen/mean:26877.375 - actor/entropy:0.1653793752193451 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9992972612380981 - training/rollout_probs_diff_mean:0.003593530971556902 - training/rollout_probs_diff_std:0.01221021730452776 - training/rollout_actor_probs_pearson_corr:0.9982529878616333 - rollout_corr/rollout_is_mean:1.0001059770584106 - rollout_corr/rollout_is_ratio_fraction_high:2.7500669602886774e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.18765116087161e-05 - rollout_corr/rollout_is_max:3.486543655395508 - rollout_corr/rollout_is_min:0.00043065936188213527 - rollout_corr/rollout_is_std:0.034376177936792374 - rollout_corr/rollout_is_eff_sample_size:0.9988199109866871 - rollout_corr/rollout_is_seq_mean:1.0000687837600708 - rollout_corr/rollout_is_seq_std:0.0017359366174787283 - rollout_corr/rollout_is_seq_max:1.0058130025863647 - rollout_corr/rollout_is_seq_min:0.9936341047286987 - rollout_corr/rollout_is_seq_max_deviation:0.0063658952713012695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1735385786741972) - rollout_corr/training_log_ppl:np.float64(0.1555803755109082) - rollout_corr/kl:np.float64(0.0008435647158400172) - rollout_corr/k3_kl:np.float64(0.000810335308230492) - rollout_corr/rollout_ppl:np.float64(1.172548221424222) - rollout_corr/rollout_log_ppl:np.float64(0.15473681054572808) - rollout_corr/log_ppl_diff:np.float64(0.0008435649651801214) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016919265181059018) - rollout_corr/log_ppl_diff_max:np.float64(0.013718657195568085) - rollout_corr/log_ppl_diff_min:np.float64(-0.005128346383571625) - rollout_corr/ppl_ratio:np.float64(1.000846435315907) - rollout_corr/chi2_token:np.float64(0.001545611536130309) - rollout_corr/chi2_seq:np.float64(0.7387706565205008) - actor/pg_loss:np.float64(-7.304479368031025e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014301339112989808) - actor/ppo_kl:np.float64(0.00022386275868435668) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4195772558450699) - perf/mfu/actor:np.float64(0.07791174639008124) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.34968948364258) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014247066341340542 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014247066341340542 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:568.16796875 - response_length/max:1852.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:568.16796875 - response_length_non_aborted/max:1852.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003154464066028595 - timing_s/agent_loop/generate_sequences/min:np.float64(1.197395345196128) - timing_s/agent_loop/generate_sequences/max:np.float64(13.042586963623762) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.787129377022211) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.042586963623762) - timi
(TaskRunner pid=2049544) Training Progress: 23%|██▎ | 23/100 [16:30<56:36, 44.11s/it]
(TaskRunner pid=2049544) step:24 - global_seqlen/min:18041 - global_seqlen/max:30779 - global_seqlen/minmax_diff:12738 - global_seqlen/balanced_min:24022 - global_seqlen/balanced_max:24069 - global_seqlen/mean:24056.125 - actor/entropy:0.143111914396286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8866641521453857 - training/rollout_probs_diff_mean:0.003361048409715295 - training/rollout_probs_diff_std:0.012565950863063335 - training/rollout_actor_probs_pearson_corr:0.9978638887405396 - rollout_corr/rollout_is_mean:1.0000112056732178 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00013656703231390566 - rollout_corr/rollout_is_max:1.9401601552963257 - rollout_corr/rollout_is_min:0.01988193951547146 - rollout_corr/rollout_is_std:0.03432759642601013 - rollout_corr/rollout_is_eff_sample_size:0.9988230031268704 - rollout_corr/rollout_is_seq_mean:1.000041127204895 - rollout_corr/rollout_is_seq_std:0.0024783890694379807 - rollout_corr/rollout_is_seq_max:1.0213310718536377 - rollout_corr/rollout_is_seq_min:0.9930093884468079 - rollout_corr/rollout_is_seq_max_deviation:0.021331071853637695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.144452238921076) - rollout_corr/training_log_ppl:np.float64(0.13059414884628495) - rollout_corr/kl:np.float64(0.00082983940849779) - rollout_corr/k3_kl:np.float64(0.0008679421140129762) - rollout_corr/rollout_ppl:np.float64(1.1434855586849153) - rollout_corr/rollout_log_ppl:np.float64(0.12976430927665206) - rollout_corr/log_ppl_diff:np.float64(0.0008298395696328953) - rollout_corr/log_ppl_abs_diff:np.float64(0.001792395327356644) - rollout_corr/log_ppl_diff_max:np.float64(0.01518431305885315) - rollout_corr/log_ppl_diff_min:np.float64(-0.016911908984184265) - rollout_corr/ppl_ratio:np.float64(1.000833987724036) - rollout_corr/chi2_token:np.float64(0.0022546271793544292) - rollout_corr/chi2_seq:np.float64(1.743649244075641) - actor/pg_loss:np.float64(0.00021742633543908596) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007519598264025262) - actor/ppo_kl:np.float64(9.309058857098762e-05) - actor/pg_clipfrac_lower:np.float64(5.556543328566477e-06) - actor/grad_norm:np.float64(0.3718913719058037) - perf/mfu/actor:np.float64(0.06935155745712235) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.33251953125) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010280685499310493 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010280685499310493 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:486.25390625 - response_length/max:1963.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:486.25390625 - response_length_non_aborted/max:1963.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013114511966705322 - timing_s/agent_loop/generate_sequences/min:np.float64(1.162827044725418) - timing_s/agent_loop/generate_sequences/max:np.float64(12.901960568502545) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.848180819972185) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.901960568502545) - timing_s/agent_loo
(TaskRunner pid=2049544) Training Progress: 24%|██▍ | 24/100 [17:02<51:12, 40.43s/it]
(TaskRunner pid=2049544) step:25 - global_seqlen/min:17209 - global_seqlen/max:32556 - global_seqlen/minmax_diff:15347 - global_seqlen/balanced_min:25066 - global_seqlen/balanced_max:25097 - global_seqlen/mean:25084.5 - actor/entropy:0.16078905761241913 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7048856616020203 - training/rollout_probs_diff_mean:0.003612553235143423 - training/rollout_probs_diff_std:0.01245187222957611 - training/rollout_actor_probs_pearson_corr:0.9980130791664124 - rollout_corr/rollout_is_mean:1.0001463890075684 - rollout_corr/rollout_is_ratio_fraction_high:3.095304418820888e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.73826104705222e-05 - rollout_corr/rollout_is_max:2.4638538360595703 - rollout_corr/rollout_is_min:0.07782533764839172 - rollout_corr/rollout_is_std:0.03481380268931389 - rollout_corr/rollout_is_eff_sample_size:0.9987899420046099 - rollout_corr/rollout_is_seq_mean:1.0001909732818604 - rollout_corr/rollout_is_seq_std:0.0020113997161388397 - rollout_corr/rollout_is_seq_max:1.0118286609649658 - rollout_corr/rollout_is_seq_min:0.9921811819076538 - rollout_corr/rollout_is_seq_max_deviation:0.01182866096496582 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1733281780034304) - rollout_corr/training_log_ppl:np.float64(0.15433276955445763) - rollout_corr/kl:np.float64(0.0005177396068951623) - rollout_corr/k3_kl:np.float64(0.0007405701547043009) - rollout_corr/rollout_ppl:np.float64(1.172693099360913) - rollout_corr/rollout_log_ppl:np.float64(0.15381502895615995) - rollout_corr/log_ppl_diff:np.float64(0.0005177405982976779) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015419888804899529) - rollout_corr/log_ppl_diff_max:np.float64(0.017087146639823914) - rollout_corr/log_ppl_diff_min:np.float64(-0.010565042495727539) - rollout_corr/ppl_ratio:np.float64(1.0005205699708313) - rollout_corr/chi2_token:np.float64(0.0020056585781276226) - rollout_corr/chi2_seq:np.float64(0.9553266488946974) - actor/pg_loss:np.float64(7.516134064644575e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005029199624004832) - actor/ppo_kl:np.float64(6.852521258160493e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3206869065761566) - perf/mfu/actor:np.float64(0.07269379031279649) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.39129257202148) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005469983443617821 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005469983443617821 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:504.796875 - response_length/max:1635.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:504.796875 - response_length_non_aborted/max:1635.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001018233597278595 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8177736550569534) - timing_s/agent_loop/generate_sequences/max:np.float64(11.67398145236075) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.060982199414866) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.67398145236075) - timing_s/agent_
(TaskRunner pid=2049544) Training Progress: 25%|██▌ | 25/100 [17:32<46:43, 37.39s/it]
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) step:26 - global_seqlen/min:19588 - global_seqlen/max:26583 - global_seqlen/minmax_diff:6995 - global_seqlen/balanced_min:22329 - global_seqlen/balanced_max:22375 - global_seqlen/mean:22358.5 - actor/entropy:0.14037299156188965 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3580157160758972 - training/rollout_probs_diff_mean:0.003176919650286436 - training/rollout_probs_diff_std:0.011454894207417965 - training/rollout_actor_probs_pearson_corr:0.9981624484062195 - rollout_corr/rollout_is_mean:1.0003305673599243 - rollout_corr/rollout_is_ratio_fraction_high:2.6683745090849698e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6683745090849698e-05 - rollout_corr/rollout_is_max:3.044771909713745 - rollout_corr/rollout_is_min:0.3202765882015228 - rollout_corr/rollout_is_std:0.03229307755827904 - rollout_corr/rollout_is_eff_sample_size:0.9989589572900974 - rollout_corr/rollout_is_seq_mean:1.0003631114959717 - rollout_corr/rollout_is_seq_std:0.001997264800593257 - rollout_corr/rollout_is_seq_max:1.009385347366333 - rollout_corr/rollout_is_seq_min:0.9931842684745789 - rollout_corr/rollout_is_seq_max_deviation:0.009385347366333008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.137321101501584) - rollout_corr/training_log_ppl:np.float64(0.12318833442259347) - rollout_corr/kl:np.float64(0.00036465673569097135) - rollout_corr/k3_kl:np.float64(0.0006004141492397252) - rollout_corr/rollout_ppl:np.float64(1.136900127865374) - rollout_corr/rollout_log_ppl:np.float64(0.12282367694933782) - rollout_corr/log_ppl_diff:np.float64(0.00036465747325564735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014688569026475307) - rollout_corr/log_ppl_diff_max:np.float64(0.011015359312295914) - rollout_corr/log_ppl_diff_min:np.float64(-0.0049578845500946045) - rollout_corr/ppl_ratio:np.float64(1.0003668149001896) - rollout_corr/chi2_token:np.float64(0.001733903307467699) - rollout_corr/chi2_seq:np.float64(0.5194167881272733) - actor/pg_loss:np.float64(8.537911344319582e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003864719383273041) - actor/ppo_kl:np.float64(0.00016390427055512635) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1358119323849678) - perf/mfu/actor:np.float64(0.06507624815359347) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.42486190795898) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01127944141626358 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01127944141626358 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:439.171875 - response_length/max:1781.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:439.171875 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013301707804203033 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8018740545958281) - timing_s/agent_loop/generate_sequences/max:np.float64(11.262868203222752) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.332270985694777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.262868203222752) - timing_s/age
(TaskRunner pid=2049544) Training Progress: 26%|██▌ | 26/100 [18:02<43:24, 35.20s/it]
(TaskRunner pid=2049544) step:27 - global_seqlen/min:20390 - global_seqlen/max:35494 - global_seqlen/minmax_diff:15104 - global_seqlen/balanced_min:27757 - global_seqlen/balanced_max:27828 - global_seqlen/mean:27799.5 - actor/entropy:0.13566820323467255 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4101554751396179 - training/rollout_probs_diff_mean:0.002812632592394948 - training/rollout_probs_diff_std:0.010530136525630951 - training/rollout_actor_probs_pearson_corr:0.9983336925506592 - rollout_corr/rollout_is_mean:1.0000394582748413 - rollout_corr/rollout_is_ratio_fraction_high:1.9780045477091335e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.615344005287625e-05 - rollout_corr/rollout_is_max:2.5563464164733887 - rollout_corr/rollout_is_min:0.3321341276168823 - rollout_corr/rollout_is_std:0.03021036833524704 - rollout_corr/rollout_is_eff_sample_size:0.9990882848714958 - rollout_corr/rollout_is_seq_mean:1.0000697374343872 - rollout_corr/rollout_is_seq_std:0.0016781107988208532 - rollout_corr/rollout_is_seq_max:1.0076134204864502 - rollout_corr/rollout_is_seq_min:0.9905708432197571 - rollout_corr/rollout_is_seq_max_deviation:0.00942915678024292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1346074021421373) - rollout_corr/training_log_ppl:np.float64(0.12218492220563348) - rollout_corr/kl:np.float64(0.0005914641850237601) - rollout_corr/k3_kl:np.float64(0.0005361424209464616) - rollout_corr/rollout_ppl:np.float64(1.1339335390366614) - rollout_corr/rollout_log_ppl:np.float64(0.1215934576903237) - rollout_corr/log_ppl_diff:np.float64(0.0005914645153097808) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011890121677424759) - rollout_corr/log_ppl_diff_max:np.float64(0.012660838663578033) - rollout_corr/log_ppl_diff_min:np.float64(-0.006112679839134216) - rollout_corr/ppl_ratio:np.float64(1.0005930345505476) - rollout_corr/chi2_token:np.float64(0.0009534833952784538) - rollout_corr/chi2_seq:np.float64(1.2373452726751566) - actor/pg_loss:np.float64(0.00019052473362535238) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004077799751485145) - actor/ppo_kl:np.float64(0.00015187378462222512) - actor/pg_clipfrac_lower:np.float64(2.895663556046202e-06) - actor/grad_norm:np.float64(0.24880316480994225) - perf/mfu/actor:np.float64(0.08016295921139835) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.3461446762085) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011563085718080401 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011563085718080401 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:592.453125 - response_length/max:2410.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:592.453125 - response_length_non_aborted/max:2410.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010118260979652405 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9887365885078907) - timing_s/agent_loop/generate_sequences/max:np.float64(15.905478810891509) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.845138130702253) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.9054788108915
(TaskRunner pid=2049544) Training Progress: 27%|██▋ | 27/100 [18:37<42:40, 35.08s/it]
(TaskRunner pid=2049544) step:28 - global_seqlen/min:17957 - global_seqlen/max:34244 - global_seqlen/minmax_diff:16287 - global_seqlen/balanced_min:25513 - global_seqlen/balanced_max:25550 - global_seqlen/mean:25526.75 - actor/entropy:0.16894282400608063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47068703174591064 - training/rollout_probs_diff_mean:0.003456175560131669 - training/rollout_probs_diff_std:0.011879513040184975 - training/rollout_actor_probs_pearson_corr:0.9982811212539673 - rollout_corr/rollout_is_mean:0.9999144077301025 - rollout_corr/rollout_is_ratio_fraction_high:1.4930944416846614e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.972377766738646e-05 - rollout_corr/rollout_is_max:2.6947081089019775 - rollout_corr/rollout_is_min:0.39785826206207275 - rollout_corr/rollout_is_std:0.03413606435060501 - rollout_corr/rollout_is_eff_sample_size:0.9988359665383245 - rollout_corr/rollout_is_seq_mean:0.9999093413352966 - rollout_corr/rollout_is_seq_std:0.002044897060841322 - rollout_corr/rollout_is_seq_max:1.0098328590393066 - rollout_corr/rollout_is_seq_min:0.9918521642684937 - rollout_corr/rollout_is_seq_max_deviation:0.00983285903930664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1787242372520268) - rollout_corr/training_log_ppl:np.float64(0.15832054920610972) - rollout_corr/kl:np.float64(0.0006220183064016283) - rollout_corr/k3_kl:np.float64(0.0006294625401039866) - rollout_corr/rollout_ppl:np.float64(1.1779772732406855) - rollout_corr/rollout_log_ppl:np.float64(0.15769852932862705) - rollout_corr/log_ppl_diff:np.float64(0.0006220198774826713) - rollout_corr/log_ppl_abs_diff:np.float64(0.001586154270626139) - rollout_corr/log_ppl_diff_max:np.float64(0.009446874260902405) - rollout_corr/log_ppl_diff_min:np.float64(-0.006108276546001434) - rollout_corr/ppl_ratio:np.float64(1.0006244094111025) - rollout_corr/chi2_token:np.float64(0.0012534137349575758) - rollout_corr/chi2_seq:np.float64(0.5095949505921453) - actor/pg_loss:np.float64(0.0001528579741716385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005431991780824319) - actor/ppo_kl:np.float64(-0.00011134325540240742) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2734988182783127) - perf/mfu/actor:np.float64(0.07474159440863055) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.44351959228516) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018798992037773132 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018798992037773132 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:523.2421875 - response_length/max:1995.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:523.2421875 - response_length_non_aborted/max:1995.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001253858208656311 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8268284611403942) - timing_s/agent_loop/generate_sequences/max:np.float64(13.178542386740446) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.144354823620233) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.178542386740446) - tim
(TaskRunner pid=2049544) Training Progress: 28%|██▊ | 28/100 [19:09<40:57, 34.13s/it]
(TaskRunner pid=2049544) step:29 - global_seqlen/min:20947 - global_seqlen/max:33732 - global_seqlen/minmax_diff:12785 - global_seqlen/balanced_min:25386 - global_seqlen/balanced_max:25685 - global_seqlen/mean:25470.375 - actor/entropy:0.13968856632709503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5444822311401367 - training/rollout_probs_diff_mean:0.0031616955529898405 - training/rollout_probs_diff_std:0.011410399340093136 - training/rollout_actor_probs_pearson_corr:0.9981111884117126 - rollout_corr/rollout_is_mean:1.000052809715271 - rollout_corr/rollout_is_ratio_fraction_high:7.336810995184351e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.8694487961474806e-05 - rollout_corr/rollout_is_max:3.6677300930023193 - rollout_corr/rollout_is_min:0.22905343770980835 - rollout_corr/rollout_is_std:0.0324532613158226 - rollout_corr/rollout_is_eff_sample_size:0.9989480129782379 - rollout_corr/rollout_is_seq_mean:1.0001238584518433 - rollout_corr/rollout_is_seq_std:0.002332444768399 - rollout_corr/rollout_is_seq_max:1.0094679594039917 - rollout_corr/rollout_is_seq_min:0.9920924305915833 - rollout_corr/rollout_is_seq_max_deviation:0.0094679594039917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.140123389661312) - rollout_corr/training_log_ppl:np.float64(0.12807749381863687) - rollout_corr/kl:np.float64(0.000638476259126719) - rollout_corr/k3_kl:np.float64(0.0006935356665564996) - rollout_corr/rollout_ppl:np.float64(1.1393897044472396) - rollout_corr/rollout_log_ppl:np.float64(0.1274390167054662) - rollout_corr/log_ppl_diff:np.float64(0.0006384771131706657) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017721431377140107) - rollout_corr/log_ppl_diff_max:np.float64(0.011614277958869934) - rollout_corr/log_ppl_diff_min:np.float64(-0.011640861630439758) - rollout_corr/ppl_ratio:np.float64(1.0006417338736355) - rollout_corr/chi2_token:np.float64(0.0015386808663606644) - rollout_corr/chi2_seq:np.float64(0.6557973187882453) - actor/pg_loss:np.float64(8.236418943852186e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006260939494495688) - actor/ppo_kl:np.float64(0.00012735309167744013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2651471309363842) - perf/mfu/actor:np.float64(0.07338363198697957) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.38971328735352) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009649740532040596 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009649740532040596 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:532.41796875 - response_length/max:2748.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:532.41796875 - response_length_non_aborted/max:2748.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.792375981807709e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8251552730798721) - timing_s/agent_loop/generate_sequences/max:np.float64(16.657433584332466) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.999644991774403) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.657433584332466) - timing_s/agent_lo
(TaskRunner pid=2049544) Training Progress: 29%|██▉ | 29/100 [19:45<40:54, 34.57s/it]
(TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30}
(TaskRunner pid=2049544) validation generation end
(TaskRunner pid=2049544) [prompt] system
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2049544) <reasoning>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) A
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) user
(TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) A: 50 pC
(TaskRunner pid=2049544) B: 56 pC
(TaskRunner pid=2049544) C: 60 pC
(TaskRunner pid=2049544) D: 64 pC
(TaskRunner pid=2049544) Please reason step by step.
(TaskRunner pid=2049544) assistant
(TaskRunner pid=2049544) <think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) [response] <reasoning>
(TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. The formula for the electric field due to a point charge is:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) E = k \frac{q}{r^2}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) We need to solve for $ q $:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{E r^2}{k}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Substitute the values:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $):
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) The closest answer is **B: 56 pC**.
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) B
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544) [ground_truth] B
(TaskRunner pid=2049544) [score] 1.0
(TaskRunner pid=2049544) [acc] 1.0
(TaskRunner pid=2049544) [pred] B
(TaskRunner pid=2049544) [incorrect_format] 1
(TaskRunner pid=2049544) [feedback]
(TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30
(WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor
(TaskRunner pid=2049544) step:30 - global_seqlen/min:16681 - global_seqlen/max:26264 - global_seqlen/minmax_diff:9583 - global_seqlen/balanced_min:22916 - global_seqlen/balanced_max:22938 - global_seqlen/mean:22931.25 - actor/entropy:0.1647506058216095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3170890808105469 - training/rollout_probs_diff_mean:0.0035670390352606773 - training/rollout_probs_diff_std:0.01208495907485485 - training/rollout_actor_probs_pearson_corr:0.9981399178504944 - rollout_corr/rollout_is_mean:0.9999049305915833 - rollout_corr/rollout_is_ratio_fraction_high:8.544525371689815e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00011962335702264681 - rollout_corr/rollout_is_max:2.2645792961120605 - rollout_corr/rollout_is_min:0.1764460951089859 - rollout_corr/rollout_is_std:0.03402237966656685 - rollout_corr/rollout_is_eff_sample_size:0.9988435782394575 - rollout_corr/rollout_is_seq_mean:0.9999919533729553 - rollout_corr/rollout_is_seq_std:0.0021591568365693092 - rollout_corr/rollout_is_seq_max:1.0069876909255981 - rollout_corr/rollout_is_seq_min:0.9930654764175415 - rollout_corr/rollout_is_seq_max_deviation:0.0069876909255981445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1598813771270216) - rollout_corr/training_log_ppl:np.float64(0.14310977509740042) - rollout_corr/kl:np.float64(0.0006737301786081673) - rollout_corr/k3_kl:np.float64(0.0006828237883240718) - rollout_corr/rollout_ppl:np.float64(1.1591023579239845) - rollout_corr/rollout_log_ppl:np.float64(0.14243604317016434) - rollout_corr/log_ppl_diff:np.float64(0.0006737319272360764) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017324861764791422) - rollout_corr/log_ppl_diff_max:np.float64(0.0082569420337677) - rollout_corr/log_ppl_diff_min:np.float64(-0.005524754524230957) - rollout_corr/ppl_ratio:np.float64(1.000676429597661) - rollout_corr/chi2_token:np.float64(0.0013968497514724731) - rollout_corr/chi2_seq:np.float64(0.42728084372356534) - actor/pg_loss:np.float64(2.553162630647421e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004172229162122676) - actor/ppo_kl:np.float64(2.4439394231023925e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29012734070420265) - perf/mfu/actor:np.float64(0.06612954074679292) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.5312614440918) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63125) - val-aux/sciknoweval/reward/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09227713650829415) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.81775) - val-aux/sciknoweval/reward/best@16/std:np.floa
(TaskRunner pid=2049544) Training Progress: 30%|███ | 30/100 [22:05<1:17:09, 66.13s/it]
(TaskRunner pid=2049544) step:31 - global_seqlen/min:19206 - global_seqlen/max:40437 - global_seqlen/minmax_diff:21231 - global_seqlen/balanced_min:26988 - global_seqlen/balanced_max:27006 - global_seqlen/mean:26994.75 - actor/entropy:0.15669088065624237 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2584149241447449 - training/rollout_probs_diff_mean:0.003061925061047077 - training/rollout_probs_diff_std:0.010589882731437683 - training/rollout_actor_probs_pearson_corr:0.9985439777374268 - rollout_corr/rollout_is_mean:1.0000404119491577 - rollout_corr/rollout_is_ratio_fraction_high:1.3810819837090094e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.7621639674180187e-05 - rollout_corr/rollout_is_max:2.6365303993225098 - rollout_corr/rollout_is_min:0.22984786331653595 - rollout_corr/rollout_is_std:0.030734580010175705 - rollout_corr/rollout_is_eff_sample_size:0.9990563960283122 - rollout_corr/rollout_is_seq_mean:1.0000174045562744 - rollout_corr/rollout_is_seq_std:0.0015726868296042085 - rollout_corr/rollout_is_seq_max:1.0063960552215576 - rollout_corr/rollout_is_seq_min:0.9952640533447266 - rollout_corr/rollout_is_seq_max_deviation:0.006396055221557617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1623710673302412) - rollout_corr/training_log_ppl:np.float64(0.14397141717927298) - rollout_corr/kl:np.float64(0.0004526557635822215) - rollout_corr/k3_kl:np.float64(0.0005084346583146271) - rollout_corr/rollout_ppl:np.float64(1.1618167613632977) - rollout_corr/rollout_log_ppl:np.float64(0.1435187609968125) - rollout_corr/log_ppl_diff:np.float64(0.0004526561824604869) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012202383077237755) - rollout_corr/log_ppl_diff_max:np.float64(0.005235195159912109) - rollout_corr/log_ppl_diff_min:np.float64(-0.004862427711486816) - rollout_corr/ppl_ratio:np.float64(1.0004539461806417) - rollout_corr/chi2_token:np.float64(0.0011370060965418816) - rollout_corr/chi2_seq:np.float64(0.43574777944013476) - actor/pg_loss:np.float64(1.602328848093748e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027079670348939544) - actor/ppo_kl:np.float64(-1.6756775252702028e-05) - actor/pg_clipfrac_lower:np.float64(2.4157391180779086e-06) - actor/grad_norm:np.float64(0.20888379216194153) - perf/mfu/actor:np.float64(0.07792531348505069) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.11697006225586) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006074171047657728 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.006074171047657728 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:565.6796875 - response_length/max:1784.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:565.6796875 - response_length_non_aborted/max:1784.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.553879499435425e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7227725125849247) - timing_s/agent_loop/generate_sequences/max:np.float64(13.10448818653822) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.673034100342193) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.1044
(TaskRunner pid=2049544) Training Progress: 31%|███ | 31/100 [22:36<1:04:13, 55.85s/it]
(TaskRunner pid=2049544) step:32 - global_seqlen/min:15472 - global_seqlen/max:26624 - global_seqlen/minmax_diff:11152 - global_seqlen/balanced_min:21157 - global_seqlen/balanced_max:21170 - global_seqlen/mean:21164.125 - actor/entropy:0.16215530037879944 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35835716128349304 - training/rollout_probs_diff_mean:0.0033622696064412594 - training/rollout_probs_diff_std:0.011649704538285732 - training/rollout_actor_probs_pearson_corr:0.9983426928520203 - rollout_corr/rollout_is_mean:0.9998896718025208 - rollout_corr/rollout_is_ratio_fraction_high:1.9218758097849786e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.765627429354936e-05 - rollout_corr/rollout_is_max:2.1169991493225098 - rollout_corr/rollout_is_min:0.016281770542263985 - rollout_corr/rollout_is_std:0.033417847007513046 - rollout_corr/rollout_is_eff_sample_size:0.9988841362904098 - rollout_corr/rollout_is_seq_mean:0.9999381303787231 - rollout_corr/rollout_is_seq_std:0.00247986800968647 - rollout_corr/rollout_is_seq_max:1.011742353439331 - rollout_corr/rollout_is_seq_min:0.9914096593856812 - rollout_corr/rollout_is_seq_max_deviation:0.011742353439331055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1614093114621937) - rollout_corr/training_log_ppl:np.float64(0.1426902024759329) - rollout_corr/kl:np.float64(0.0009178641532645582) - rollout_corr/k3_kl:np.float64(0.0007639187314083529) - rollout_corr/rollout_ppl:np.float64(1.1602927432395518) - rollout_corr/rollout_log_ppl:np.float64(0.1417723370686872) - rollout_corr/log_ppl_diff:np.float64(0.0009178654072456993) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020935917491442524) - rollout_corr/log_ppl_diff_max:np.float64(0.03533336520195007) - rollout_corr/log_ppl_diff_min:np.float64(-0.009082436561584473) - rollout_corr/ppl_ratio:np.float64(1.000924628926441) - rollout_corr/chi2_token:np.float64(0.00115761230699718) - rollout_corr/chi2_seq:np.float64(0.4135602100286633) - actor/pg_loss:np.float64(-7.768673822283745e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003810818443525932) - actor/ppo_kl:np.float64(0.00014974098553555137) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26316603645682335) - perf/mfu/actor:np.float64(0.0603751772224912) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.58000946044922) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003006534418091178 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003006534418091178 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.50390625 - response_length/max:1261.0 - response_length/min:50.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.50390625 - response_length_non_aborted/max:1261.0 - response_length_non_aborted/min:50.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013849511742591858 - timing_s/agent_loop/generate_sequences/min:np.float64(0.584519986063242) - timing_s/agent_loop/generate_sequences/max:np.float64(9.110312020406127) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.030619656012277) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.110312020406127) - timing_
(TaskRunner pid=2049544) Training Progress: 32%|███▏ | 32/100 [23:04<53:42, 47.39s/it]
(TaskRunner pid=2049544) step:33 - global_seqlen/min:21009 - global_seqlen/max:35793 - global_seqlen/minmax_diff:14784 - global_seqlen/balanced_min:26523 - global_seqlen/balanced_max:26619 - global_seqlen/mean:26546.375 - actor/entropy:0.1607956439256668 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4141917824745178 - training/rollout_probs_diff_mean:0.0031411326490342617 - training/rollout_probs_diff_std:0.01075077150017023 - training/rollout_actor_probs_pearson_corr:0.998579204082489 - rollout_corr/rollout_is_mean:1.0000613927841187 - rollout_corr/rollout_is_ratio_fraction_high:2.17348788282834e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.0714716053334996e-05 - rollout_corr/rollout_is_max:5.24385929107666 - rollout_corr/rollout_is_min:0.3405860662460327 - rollout_corr/rollout_is_std:0.0315140075981617 - rollout_corr/rollout_is_eff_sample_size:0.9990078527425291 - rollout_corr/rollout_is_seq_mean:1.0000804662704468 - rollout_corr/rollout_is_seq_std:0.0017564821755513549 - rollout_corr/rollout_is_seq_max:1.006698489189148 - rollout_corr/rollout_is_seq_min:0.995316743850708 - rollout_corr/rollout_is_seq_max_deviation:0.006698489189147949 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.176677621435374) - rollout_corr/training_log_ppl:np.float64(0.15545551202376373) - rollout_corr/kl:np.float64(0.000548688222613336) - rollout_corr/k3_kl:np.float64(0.0006223271676191189) - rollout_corr/rollout_ppl:np.float64(1.1760094286873937) - rollout_corr/rollout_log_ppl:np.float64(0.1549068232634454) - rollout_corr/log_ppl_diff:np.float64(0.0005486887603183277) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014138846672722138) - rollout_corr/log_ppl_diff_max:np.float64(0.008264932781457901) - rollout_corr/log_ppl_diff_min:np.float64(-0.005941852927207947) - rollout_corr/ppl_ratio:np.float64(1.0005506034940481) - rollout_corr/chi2_token:np.float64(0.0014625547919422388) - rollout_corr/chi2_seq:np.float64(0.45368791883811355) - actor/pg_loss:np.float64(5.0174305215477943e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003640777423470354) - actor/ppo_kl:np.float64(6.6643967455704e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2803301140666008) - perf/mfu/actor:np.float64(0.07602373904107226) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.56497955322266) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006237910129129887 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006237910129129887 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:539.16796875 - response_length/max:2340.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:539.16796875 - response_length_non_aborted/max:2340.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011204741895198822 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2724983282387257) - timing_s/agent_loop/generate_sequences/max:np.float64(14.548386441543698) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.296186965904781) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.548386441543698) - timing_s/agen
(TaskRunner pid=2049544) Training Progress: 33%|███▎ | 33/100 [23:38<48:17, 43.24s/it]
(TaskRunner pid=2049544) step:34 - global_seqlen/min:19210 - global_seqlen/max:32362 - global_seqlen/minmax_diff:13152 - global_seqlen/balanced_min:24672 - global_seqlen/balanced_max:24722 - global_seqlen/mean:24706.625 - actor/entropy:0.1634763479232788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.003025032114237547 - training/rollout_probs_diff_std:0.010739969089627266 - training/rollout_actor_probs_pearson_corr:0.998622715473175 - rollout_corr/rollout_is_mean:1.0000046491622925 - rollout_corr/rollout_is_ratio_fraction_high:2.2830181478639133e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.84905462549068e-05 - rollout_corr/rollout_is_max:3.96623158454895 - rollout_corr/rollout_is_min:0.17613501846790314 - rollout_corr/rollout_is_std:0.03082752786576748 - rollout_corr/rollout_is_eff_sample_size:0.9990505658256099 - rollout_corr/rollout_is_seq_mean:1.0001435279846191 - rollout_corr/rollout_is_seq_std:0.001950009143911302 - rollout_corr/rollout_is_seq_max:1.0084269046783447 - rollout_corr/rollout_is_seq_min:0.9940277338027954 - rollout_corr/rollout_is_seq_max_deviation:0.008426904678344727 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1493624411523342) - rollout_corr/training_log_ppl:np.float64(0.13525041530374438) - rollout_corr/kl:np.float64(0.0005815880843990096) - rollout_corr/k3_kl:np.float64(0.0006954628924091821) - rollout_corr/rollout_ppl:np.float64(1.1486861389130354) - rollout_corr/rollout_log_ppl:np.float64(0.13466882749344222) - rollout_corr/log_ppl_diff:np.float64(0.0005815878103021532) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016140752850333229) - rollout_corr/log_ppl_diff_max:np.float64(0.00731133297085762) - rollout_corr/log_ppl_diff_min:np.float64(-0.007330030202865601) - rollout_corr/ppl_ratio:np.float64(1.0005840898957103) - rollout_corr/chi2_token:np.float64(0.0018483358435332775) - rollout_corr/chi2_seq:np.float64(0.8693230981007218) - actor/pg_loss:np.float64(-7.774424739181995e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004309596388338832) - actor/ppo_kl:np.float64(0.00016126566817398214) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24917255342006683) - perf/mfu/actor:np.float64(0.07059737130291072) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.48730087280273) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002149841981008649 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002149841981008649 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:513.30078125 - response_length/max:2220.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:513.30078125 - response_length_non_aborted/max:2220.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001704581081867218 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9642345681786537) - timing_s/agent_loop/generate_sequences/max:np.float64(14.358597544953227) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.930665408472123) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.358597544953227) - timing_s/age
(TaskRunner pid=2049544) Training Progress: 34%|███▍ | 34/100 [24:11<44:13, 40.21s/it]
(TaskRunner pid=2049544) step:35 - global_seqlen/min:18610 - global_seqlen/max:31076 - global_seqlen/minmax_diff:12466 - global_seqlen/balanced_min:24486 - global_seqlen/balanced_max:24516 - global_seqlen/mean:24505.375 - actor/entropy:0.18308240175247192 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6351485848426819 - training/rollout_probs_diff_mean:0.003324724268168211 - training/rollout_probs_diff_std:0.011287217028439045 - training/rollout_actor_probs_pearson_corr:0.998609185218811 - rollout_corr/rollout_is_mean:0.9998504519462585 - rollout_corr/rollout_is_ratio_fraction_high:1.5409863408422098e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.852466034004465e-05 - rollout_corr/rollout_is_max:3.141425609588623 - rollout_corr/rollout_is_min:0.22313041985034943 - rollout_corr/rollout_is_std:0.03237971290946007 - rollout_corr/rollout_is_eff_sample_size:0.9989524144661357 - rollout_corr/rollout_is_seq_mean:0.999904215335846 - rollout_corr/rollout_is_seq_std:0.002002937952056527 - rollout_corr/rollout_is_seq_max:1.0084916353225708 - rollout_corr/rollout_is_seq_min:0.9936511516571045 - rollout_corr/rollout_is_seq_max_deviation:0.0084916353225708 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1920651025138795) - rollout_corr/training_log_ppl:np.float64(0.16792560920293909) - rollout_corr/kl:np.float64(0.0008154499521539904) - rollout_corr/k3_kl:np.float64(0.0007078099062596266) - rollout_corr/rollout_ppl:np.float64(1.1910648443736136) - rollout_corr/rollout_log_ppl:np.float64(0.1671101573156193) - rollout_corr/log_ppl_diff:np.float64(0.0008154518873197958) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017030747985700145) - rollout_corr/log_ppl_diff_max:np.float64(0.010384440422058105) - rollout_corr/log_ppl_diff_min:np.float64(-0.007758818566799164) - rollout_corr/ppl_ratio:np.float64(1.0008183948229998) - rollout_corr/chi2_token:np.float64(0.0012777894735336304) - rollout_corr/chi2_seq:np.float64(1.3958141808398068) - actor/pg_loss:np.float64(1.1038966476917267e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003179428899784398) - actor/ppo_kl:np.float64(0.00012983981464032013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2326635681092739) - perf/mfu/actor:np.float64(0.07068896275098592) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.57692337036133) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004130806773900986 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004130806773900986 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:506.98046875 - response_length/max:1617.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:506.98046875 - response_length_non_aborted/max:1617.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.149631321430206e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9240005984902382) - timing_s/agent_loop/generate_sequences/max:np.float64(11.936645423993468) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.090721758017025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.936645423993468) - timing_s
(TaskRunner pid=2049544) Training Progress: 35%|███▌ | 35/100 [24:41<40:28, 37.37s/it]
(TaskRunner pid=2049544) step:36 - global_seqlen/min:24110 - global_seqlen/max:37995 - global_seqlen/minmax_diff:13885 - global_seqlen/balanced_min:31143 - global_seqlen/balanced_max:31201 - global_seqlen/mean:31181.25 - actor/entropy:0.2063475251197815 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35195159912109375 - training/rollout_probs_diff_mean:0.003259577788412571 - training/rollout_probs_diff_std:0.010389592498540878 - training/rollout_actor_probs_pearson_corr:0.9988572597503662 - rollout_corr/rollout_is_mean:0.9999135732650757 - rollout_corr/rollout_is_ratio_fraction_high:5.54268399355351e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.217073597421404e-05 - rollout_corr/rollout_is_max:5.167435169219971 - rollout_corr/rollout_is_min:0.08687802404165268 - rollout_corr/rollout_is_std:0.03102603740990162 - rollout_corr/rollout_is_eff_sample_size:0.9990380727632393 - rollout_corr/rollout_is_seq_mean:0.9999022483825684 - rollout_corr/rollout_is_seq_std:0.0015286069829016924 - rollout_corr/rollout_is_seq_max:1.0051732063293457 - rollout_corr/rollout_is_seq_min:0.9951693415641785 - rollout_corr/rollout_is_seq_max_deviation:0.005173206329345703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2262746798805892) - rollout_corr/training_log_ppl:np.float64(0.19696971234225202) - rollout_corr/kl:np.float64(0.0006086873604012055) - rollout_corr/k3_kl:np.float64(0.0005889998031136656) - rollout_corr/rollout_ppl:np.float64(1.2254916024394333) - rollout_corr/rollout_log_ppl:np.float64(0.19636102448566817) - rollout_corr/log_ppl_diff:np.float64(0.000608687856583856) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012519802548922598) - rollout_corr/log_ppl_diff_max:np.float64(0.006760932505130768) - rollout_corr/log_ppl_diff_min:np.float64(-0.005945771932601929) - rollout_corr/ppl_ratio:np.float64(1.0006101776380092) - rollout_corr/chi2_token:np.float64(0.001163150416687131) - rollout_corr/chi2_seq:np.float64(0.9006679092999548) - actor/pg_loss:np.float64(6.59312354400754e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041949365038362885) - actor/ppo_kl:np.float64(-4.479620235997572e-05) - actor/pg_clipfrac_lower:np.float64(3.426535158723709e-06) - actor/grad_norm:np.float64(0.21542762964963913) - perf/mfu/actor:np.float64(0.08866368508784078) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.68338394165039) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03192170336842537 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03192170336842537 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:704.7578125 - response_length/max:2399.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:704.7578125 - response_length_non_aborted/max:2399.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.613484144210815e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1882787812501192) - timing_s/agent_loop/generate_sequences/max:np.float64(16.427959153428674) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.1280951405206) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.427959153
(TaskRunner pid=2049544) Training Progress: 36%|███▌ | 36/100 [25:17<39:11, 36.74s/it]
(TaskRunner pid=2049544) step:37 - global_seqlen/min:20543 - global_seqlen/max:33018 - global_seqlen/minmax_diff:12475 - global_seqlen/balanced_min:26755 - global_seqlen/balanced_max:26801 - global_seqlen/mean:26782.75 - actor/entropy:0.1873542070388794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535153567790985 - training/rollout_probs_diff_mean:0.003138389438390732 - training/rollout_probs_diff_std:0.010506313294172287 - training/rollout_actor_probs_pearson_corr:0.9987807273864746 - rollout_corr/rollout_is_mean:0.9999997019767761 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.388498225831427e-05 - rollout_corr/rollout_is_max:1.984167218208313 - rollout_corr/rollout_is_min:0.38167211413383484 - rollout_corr/rollout_is_std:0.031033720821142197 - rollout_corr/rollout_is_eff_sample_size:0.9990378348031789 - rollout_corr/rollout_is_seq_mean:0.9999757409095764 - rollout_corr/rollout_is_seq_std:0.0020235180854797363 - rollout_corr/rollout_is_seq_max:1.007972240447998 - rollout_corr/rollout_is_seq_min:0.991079568862915 - rollout_corr/rollout_is_seq_max_deviation:0.008920431137084961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.17386184213683) - rollout_corr/training_log_ppl:np.float64(0.15399431145488052) - rollout_corr/kl:np.float64(0.000585384931017785) - rollout_corr/k3_kl:np.float64(0.0005648392770467581) - rollout_corr/rollout_ppl:np.float64(1.1731591601856053) - rollout_corr/rollout_log_ppl:np.float64(0.15340892578387866) - rollout_corr/log_ppl_diff:np.float64(0.000585385671001859) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015394459333037958) - rollout_corr/log_ppl_diff_max:np.float64(0.009715557098388672) - rollout_corr/log_ppl_diff_min:np.float64(-0.006565690040588379) - rollout_corr/ppl_ratio:np.float64(1.0005877201911062) - rollout_corr/chi2_token:np.float64(0.0010866590309888124) - rollout_corr/chi2_seq:np.float64(1.2919703791849315) - actor/pg_loss:np.float64(-7.207388989627361e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027252964378021716) - actor/ppo_kl:np.float64(2.565641079144143e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2241645734757185) - perf/mfu/actor:np.float64(0.07756524385091812) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.62493515014648) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012340063229203224 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012340063229203224 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:576.3984375 - response_length/max:2563.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:576.3984375 - response_length_non_aborted/max:2563.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.650814950466156e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7121520359069109) - timing_s/agent_loop/generate_sequences/max:np.float64(16.06510110758245) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.399860835161235) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.06510110758245) - timing_s/agent_loop/slowest/tool_c
(TaskRunner pid=2049544) Training Progress: 37%|███▋ | 37/100 [25:52<38:00, 36.20s/it]
(TaskRunner pid=2049544) step:38 - global_seqlen/min:20896 - global_seqlen/max:36221 - global_seqlen/minmax_diff:15325 - global_seqlen/balanced_min:26629 - global_seqlen/balanced_max:26693 - global_seqlen/mean:26673.0 - actor/entropy:0.17777901887893677 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311151742935181 - training/rollout_probs_diff_mean:0.003014863934367895 - training/rollout_probs_diff_std:0.01037665270268917 - training/rollout_actor_probs_pearson_corr:0.998752772808075 - rollout_corr/rollout_is_mean:1.0000866651535034 - rollout_corr/rollout_is_ratio_fraction_high:2.75694746960653e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.446184564381838e-05 - rollout_corr/rollout_is_max:4.722990036010742 - rollout_corr/rollout_is_min:0.217980295419693 - rollout_corr/rollout_is_std:0.03060242161154747 - rollout_corr/rollout_is_eff_sample_size:0.9990646060209646 - rollout_corr/rollout_is_seq_mean:1.0000120401382446 - rollout_corr/rollout_is_seq_std:0.0017710411921143532 - rollout_corr/rollout_is_seq_max:1.005569338798523 - rollout_corr/rollout_is_seq_min:0.9887515902519226 - rollout_corr/rollout_is_seq_max_deviation:0.011248409748077393 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1705434750765562) - rollout_corr/training_log_ppl:np.float64(0.15039988774515223) - rollout_corr/kl:np.float64(0.0003758689842143781) - rollout_corr/k3_kl:np.float64(0.0005343400661388387) - rollout_corr/rollout_ppl:np.float64(1.170116110239178) - rollout_corr/rollout_log_ppl:np.float64(0.1500240175955696) - rollout_corr/log_ppl_diff:np.float64(0.00037587014958262444) - rollout_corr/log_ppl_abs_diff:np.float64(0.00122680542699527) - rollout_corr/log_ppl_diff_max:np.float64(0.012495182454586029) - rollout_corr/log_ppl_diff_min:np.float64(-0.005860418081283569) - rollout_corr/ppl_ratio:np.float64(1.000377441989258) - rollout_corr/chi2_token:np.float64(0.0015310256276279688) - rollout_corr/chi2_seq:np.float64(0.8941571062896401) - actor/pg_loss:np.float64(7.285538595169783e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002832129719081422) - actor/ppo_kl:np.float64(-9.715641379060003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20185822248458862) - perf/mfu/actor:np.float64(0.07681988024557596) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.53835678100586) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021598960738629103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021598960738629103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:566.75 - response_length/max:2166.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:566.75 - response_length_non_aborted/max:2166.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.976459503173828e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1030525546520948) - timing_s/agent_loop/generate_sequences/max:np.float64(14.58826527185738) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.473840224818559) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.58826527185738) - timing_s/agent_loop/slow
(TaskRunner pid=2049544) Training Progress: 38%|███▊ | 38/100 [26:25<36:34, 35.40s/it]
(TaskRunner pid=2049544) step:39 - global_seqlen/min:15255 - global_seqlen/max:27450 - global_seqlen/minmax_diff:12195 - global_seqlen/balanced_min:22238 - global_seqlen/balanced_max:22271 - global_seqlen/mean:22258.375 - actor/entropy:0.1585860699415207 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.403981477022171 - training/rollout_probs_diff_mean:0.003201692830771208 - training/rollout_probs_diff_std:0.011194097809493542 - training/rollout_actor_probs_pearson_corr:0.9984230995178223 - rollout_corr/rollout_is_mean:1.0000526905059814 - rollout_corr/rollout_is_ratio_fraction_high:4.465760866878554e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.252065213629976e-05 - rollout_corr/rollout_is_max:3.0618841648101807 - rollout_corr/rollout_is_min:0.25154978036880493 - rollout_corr/rollout_is_std:0.032896533608436584 - rollout_corr/rollout_is_eff_sample_size:0.9989191068729061 - rollout_corr/rollout_is_seq_mean:1.0000567436218262 - rollout_corr/rollout_is_seq_std:0.0022173086181282997 - rollout_corr/rollout_is_seq_max:1.0083491802215576 - rollout_corr/rollout_is_seq_min:0.9902656674385071 - rollout_corr/rollout_is_seq_max_deviation:0.00973433256149292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1577396178618073) - rollout_corr/training_log_ppl:np.float64(0.13937123074720148) - rollout_corr/kl:np.float64(0.00047606502702546294) - rollout_corr/k3_kl:np.float64(0.0006253823398196801) - rollout_corr/rollout_ppl:np.float64(1.157200226560235) - rollout_corr/rollout_log_ppl:np.float64(0.1388951658955193) - rollout_corr/log_ppl_diff:np.float64(0.0004760648516821675) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017094570139306597) - rollout_corr/log_ppl_diff_max:np.float64(0.008976280689239502) - rollout_corr/log_ppl_diff_min:np.float64(-0.007652416825294495) - rollout_corr/ppl_ratio:np.float64(1.0004788676742464) - rollout_corr/chi2_token:np.float64(0.001593738328665495) - rollout_corr/chi2_seq:np.float64(1.153579653473571) - actor/pg_loss:np.float64(-4.086887929588556e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021084232048451668) - actor/ppo_kl:np.float64(-5.276580297675082e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22345420345664024) - perf/mfu/actor:np.float64(0.06472844108846296) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.6031494140625) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0039901575073599815 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0039901575073599815 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:437.35546875 - response_length/max:1579.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.35546875 - response_length_non_aborted/max:1579.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015556812286376953 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7400109507143497) - timing_s/agent_loop/generate_sequences/max:np.float64(10.687160091474652) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.344982730093761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.687160091474652) - ti
(TaskRunner pid=2049544) Training Progress: 39%|███▉ | 39/100 [26:54<34:04, 33.52s/it]
(TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40}
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) validation generation end
(TaskRunner pid=2049544) [prompt] system
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2049544) <reasoning>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) A
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) user
(TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) A: 50 pC
(TaskRunner pid=2049544) B: 56 pC
(TaskRunner pid=2049544) C: 60 pC
(TaskRunner pid=2049544) D: 64 pC
(TaskRunner pid=2049544) Please reason step by step.
(TaskRunner pid=2049544) assistant
(TaskRunner pid=2049544) <think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) [response] <reasoning>
(TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) E = k \frac{q}{r^2}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Rearranging to solve for $ q $:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{E r^2}{k}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Substitute the values:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $):
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) The closest answer is **B: 56 pC**.
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) B
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544) [ground_truth] B
(TaskRunner pid=2049544) [score] 1.0
(TaskRunner pid=2049544) [acc] 1.0
(TaskRunner pid=2049544) [pred] B
(TaskRunner pid=2049544) [incorrect_format] 1
(TaskRunner pid=2049544) [feedback]
(TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40
(WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor
(TaskRunner pid=2049544) step:40 - global_seqlen/min:20667 - global_seqlen/max:43425 - global_seqlen/minmax_diff:22758 - global_seqlen/balanced_min:28994 - global_seqlen/balanced_max:29096 - global_seqlen/mean:29057.375 - actor/entropy:0.16825498640537262 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38080501556396484 - training/rollout_probs_diff_mean:0.002904911758378148 - training/rollout_probs_diff_std:0.010255875997245312 - training/rollout_actor_probs_pearson_corr:0.9987383484840393 - rollout_corr/rollout_is_mean:0.9999452829360962 - rollout_corr/rollout_is_ratio_fraction_high:1.811298898246605e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.62259779649321e-05 - rollout_corr/rollout_is_max:2.355332374572754 - rollout_corr/rollout_is_min:0.382403701543808 - rollout_corr/rollout_is_std:0.029906976968050003 - rollout_corr/rollout_is_eff_sample_size:0.9991061339931943 - rollout_corr/rollout_is_seq_mean:0.9999656081199646 - rollout_corr/rollout_is_seq_std:0.0016488394467160106 - rollout_corr/rollout_is_seq_max:1.009485125541687 - rollout_corr/rollout_is_seq_min:0.9944038391113281 - rollout_corr/rollout_is_seq_max_deviation:0.009485125541687012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.157492823433131) - rollout_corr/training_log_ppl:np.float64(0.14261198020540178) - rollout_corr/kl:np.float64(0.0005399936895650326) - rollout_corr/k3_kl:np.float64(0.0005378889725164981) - rollout_corr/rollout_ppl:np.float64(1.156866628676653) - rollout_corr/rollout_log_ppl:np.float64(0.1420719847228611) - rollout_corr/log_ppl_diff:np.float64(0.0005399954825406894) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012537419388536364) - rollout_corr/log_ppl_diff_max:np.float64(0.009929120540618896) - rollout_corr/log_ppl_diff_min:np.float64(-0.007549874484539032) - rollout_corr/ppl_ratio:np.float64(1.0005416304338723) - rollout_corr/chi2_token:np.float64(0.0010685438755899668) - rollout_corr/chi2_seq:np.float64(1.119046370498836) - actor/pg_loss:np.float64(-2.1900166757404804e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024728815151320305) - actor/ppo_kl:np.float64(-7.47417178104115e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1894925720989704) - perf/mfu/actor:np.float64(0.08261510768114952) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.61338424682617) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/reward/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.0950390140033621) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8010999999999999) - val-aux/sciknoweval/reward/b
(TaskRunner pid=2049544) Training Progress: 40%|████ | 40/100 [29:51<1:16:29, 76.50s/it]
(TaskRunner pid=2049544) step:41 - global_seqlen/min:24357 - global_seqlen/max:35343 - global_seqlen/minmax_diff:10986 - global_seqlen/balanced_min:29480 - global_seqlen/balanced_max:29507 - global_seqlen/mean:29501.125 - actor/entropy:0.1756914258003235 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44652730226516724 - training/rollout_probs_diff_mean:0.00312806130386889 - training/rollout_probs_diff_std:0.010616622865200043 - training/rollout_actor_probs_pearson_corr:0.9987709522247314 - rollout_corr/rollout_is_mean:0.9999130368232727 - rollout_corr/rollout_is_ratio_fraction_high:1.81861159944674e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.424815647827927e-05 - rollout_corr/rollout_is_max:3.465665340423584 - rollout_corr/rollout_is_min:0.21781140565872192 - rollout_corr/rollout_is_std:0.031423088163137436 - rollout_corr/rollout_is_eff_sample_size:0.9990134444982739 - rollout_corr/rollout_is_seq_mean:0.9999098777770996 - rollout_corr/rollout_is_seq_std:0.0014667019713670015 - rollout_corr/rollout_is_seq_max:1.0048595666885376 - rollout_corr/rollout_is_seq_min:0.9947068691253662 - rollout_corr/rollout_is_seq_max_deviation:0.005293130874633789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1796441837213933) - rollout_corr/training_log_ppl:np.float64(0.16063264349941164) - rollout_corr/kl:np.float64(0.0006431586379722631) - rollout_corr/k3_kl:np.float64(0.0005324390123746525) - rollout_corr/rollout_ppl:np.float64(1.178866429720074) - rollout_corr/rollout_log_ppl:np.float64(0.15998948468040908) - rollout_corr/log_ppl_diff:np.float64(0.000643158819002565) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012713416290353052) - rollout_corr/log_ppl_diff_max:np.float64(0.007460832595825195) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031778663396835327) - rollout_corr/ppl_ratio:np.float64(1.0006445937324315) - rollout_corr/chi2_token:np.float64(0.0008295027073472738) - rollout_corr/chi2_seq:np.float64(0.9563508839346468) - actor/pg_loss:np.float64(9.131035767495632e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002985205437653349) - actor/ppo_kl:np.float64(3.691702790131757e-05) - actor/pg_clipfrac_lower:np.float64(5.99118084210204e-06) - actor/grad_norm:np.float64(0.2258460782468319) - perf/mfu/actor:np.float64(0.08484757615781298) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.7967414855957) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01819217950105667 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01819217950105667 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:644.37890625 - response_length/max:1915.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:644.37890625 - response_length_non_aborted/max:1915.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.878321826457977e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2621874269098043) - timing_s/agent_loop/generate_sequences/max:np.float64(14.428985564038157) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.6339426440390525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.428985
(TaskRunner pid=2049544) Training Progress: 41%|████ | 41/100 [30:24<1:02:23, 63.45s/it]
(TaskRunner pid=2049544) step:42 - global_seqlen/min:25542 - global_seqlen/max:34694 - global_seqlen/minmax_diff:9152 - global_seqlen/balanced_min:30320 - global_seqlen/balanced_max:30380 - global_seqlen/mean:30366.25 - actor/entropy:0.19629985094070435 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31755661964416504 - training/rollout_probs_diff_mean:0.0030814865604043007 - training/rollout_probs_diff_std:0.010210192762315273 - training/rollout_actor_probs_pearson_corr:0.9988877177238464 - rollout_corr/rollout_is_mean:0.9999521374702454 - rollout_corr/rollout_is_ratio_fraction_high:1.7170919818454422e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.7170919818454422e-05 - rollout_corr/rollout_is_max:2.3355937004089355 - rollout_corr/rollout_is_min:0.20813822746276855 - rollout_corr/rollout_is_std:0.03025965206325054 - rollout_corr/rollout_is_eff_sample_size:0.9990850720973192 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0012307165889069438 - rollout_corr/rollout_is_seq_max:1.006462574005127 - rollout_corr/rollout_is_seq_min:0.9960255026817322 - rollout_corr/rollout_is_seq_max_deviation:0.006462574005126953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1838456057012081) - rollout_corr/training_log_ppl:np.float64(0.161605034765671) - rollout_corr/kl:np.float64(0.00040539363182290344) - rollout_corr/k3_kl:np.float64(0.0005141745206458381) - rollout_corr/rollout_ppl:np.float64(1.1833340604789555) - rollout_corr/rollout_log_ppl:np.float64(0.1611996397114126) - rollout_corr/log_ppl_diff:np.float64(0.00040539505425840616) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010806201898958534) - rollout_corr/log_ppl_diff_max:np.float64(0.005684956908226013) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034256726503372192) - rollout_corr/ppl_ratio:np.float64(1.000406396575272) - rollout_corr/chi2_token:np.float64(0.0012630913406610489) - rollout_corr/chi2_seq:np.float64(1.0963045465759933) - actor/pg_loss:np.float64(-1.263362355530262e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005835516549268505) - actor/ppo_kl:np.float64(-5.1322627289351885e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21340569853782654) - perf/mfu/actor:np.float64(0.0874389401304758) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.24371719360352) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0002196446730522439 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0002196446730522439 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:682.4765625 - response_length/max:2365.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:682.4765625 - response_length_non_aborted/max:2365.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014196522533893585 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5289256758987904) - timing_s/agent_loop/generate_sequences/max:np.float64(16.370171174407005) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.762006698663754) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.370171174407005) -
(TaskRunner pid=2049544) Training Progress: 42%|████▏ | 42/100 [31:00<53:11, 55.03s/it]
(TaskRunner pid=2049544) step:43 - global_seqlen/min:22214 - global_seqlen/max:34661 - global_seqlen/minmax_diff:12447 - global_seqlen/balanced_min:28283 - global_seqlen/balanced_max:28875 - global_seqlen/mean:28428.25 - actor/entropy:0.17894256114959717 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9968691468238831 - training/rollout_probs_diff_mean:0.0030561003368347883 - training/rollout_probs_diff_std:0.010964183136820793 - training/rollout_actor_probs_pearson_corr:0.9986260533332825 - rollout_corr/rollout_is_mean:0.9999217391014099 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6834687711670995e-05 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:3.1283660064218566e-05 - rollout_corr/rollout_is_std:0.02998857945203781 - rollout_corr/rollout_is_eff_sample_size:0.9991012551699266 - rollout_corr/rollout_is_seq_mean:0.9999701976776123 - rollout_corr/rollout_is_seq_std:0.0015368059976026416 - rollout_corr/rollout_is_seq_max:1.007220983505249 - rollout_corr/rollout_is_seq_min:0.9942285418510437 - rollout_corr/rollout_is_seq_max_deviation:0.0072209835052490234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1693742866627872) - rollout_corr/training_log_ppl:np.float64(0.15083149004203733) - rollout_corr/kl:np.float64(0.0007233850172774225) - rollout_corr/k3_kl:np.float64(0.0006608226789381888) - rollout_corr/rollout_ppl:np.float64(1.168502108193934) - rollout_corr/rollout_log_ppl:np.float64(0.15010810326202773) - rollout_corr/log_ppl_diff:np.float64(0.000723386780009605) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014408983988687396) - rollout_corr/log_ppl_diff_max:np.float64(0.025284215807914734) - rollout_corr/log_ppl_diff_min:np.float64(-0.005987722426652908) - rollout_corr/ppl_ratio:np.float64(1.0007265941239893) - rollout_corr/chi2_token:np.float64(0.0009258103091269732) - rollout_corr/chi2_seq:np.float64(1.824508806457743) - actor/pg_loss:np.float64(0.0001151182223111391) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000508402825516896) - actor/ppo_kl:np.float64(5.8405713696974004e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2429964765906334) - perf/mfu/actor:np.float64(0.08157217656976501) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.25969696044922) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018738869577646255 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018738869577646255 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:618.5703125 - response_length/max:2843.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:618.5703125 - response_length_non_aborted/max:2843.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.98380172252655e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5165256280452013) - timing_s/agent_loop/generate_sequences/max:np.float64(18.731039391830564) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.20822462200158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.731039391830564) - timing_s/agent_loop/s
(TaskRunner pid=2049544) Training Progress: 43%|████▎ | 43/100 [31:37<47:14, 49.73s/it]
(TaskRunner pid=2049544) step:44 - global_seqlen/min:22462 - global_seqlen/max:40198 - global_seqlen/minmax_diff:17736 - global_seqlen/balanced_min:31295 - global_seqlen/balanced_max:31374 - global_seqlen/mean:31349.5 - actor/entropy:0.17300812900066376 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6893051266670227 - training/rollout_probs_diff_mean:0.002969349967315793 - training/rollout_probs_diff_std:0.010275507345795631 - training/rollout_actor_probs_pearson_corr:0.9987791776657104 - rollout_corr/rollout_is_mean:0.9999350309371948 - rollout_corr/rollout_is_ratio_fraction_high:1.112074914999539e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.448299659998156e-05 - rollout_corr/rollout_is_max:2.314483165740967 - rollout_corr/rollout_is_min:0.03194784000515938 - rollout_corr/rollout_is_std:0.030064012855291367 - rollout_corr/rollout_is_eff_sample_size:0.9990967333762735 - rollout_corr/rollout_is_seq_mean:0.9999725222587585 - rollout_corr/rollout_is_seq_std:0.0014090986223891377 - rollout_corr/rollout_is_seq_max:1.0068600177764893 - rollout_corr/rollout_is_seq_min:0.9924980998039246 - rollout_corr/rollout_is_seq_max_deviation:0.0075019001960754395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1756953983567655) - rollout_corr/training_log_ppl:np.float64(0.15532817154598888) - rollout_corr/kl:np.float64(0.00048252152897987344) - rollout_corr/k3_kl:np.float64(0.0005270084845747647) - rollout_corr/rollout_ppl:np.float64(1.1751229297369719) - rollout_corr/rollout_log_ppl:np.float64(0.15484564797225175) - rollout_corr/log_ppl_diff:np.float64(0.00048252357373712584) - rollout_corr/log_ppl_abs_diff:np.float64(0.00115150140482001) - rollout_corr/log_ppl_diff_max:np.float64(0.01116035133600235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005915746092796326) - rollout_corr/ppl_ratio:np.float64(1.0004838448949158) - rollout_corr/chi2_token:np.float64(0.0010843609925359488) - rollout_corr/chi2_seq:np.float64(0.9741545785218477) - actor/pg_loss:np.float64(-0.00012030208017677069) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002879910933870633) - actor/ppo_kl:np.float64(-3.409391269992845e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15622974559664726) - perf/mfu/actor:np.float64(0.08872996761202245) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.24708938598633) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017787639051675797 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017787639051675797 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:702.515625 - response_length/max:2613.0 - response_length/min:172.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:702.515625 - response_length_non_aborted/max:2613.0 - response_length_non_aborted/min:172.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001371428370475769 - timing_s/agent_loop/generate_sequences/min:np.float64(2.046190181747079) - timing_s/agent_loop/generate_sequences/max:np.float64(18.486025908961892) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.959546060519642) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.486025908961892) - ti
(TaskRunner pid=2049544) Training Progress: 44%|████▍ | 44/100 [32:15<43:02, 46.11s/it]
(TaskRunner pid=2049544) step:45 - global_seqlen/min:20785 - global_seqlen/max:36095 - global_seqlen/minmax_diff:15310 - global_seqlen/balanced_min:29569 - global_seqlen/balanced_max:29783 - global_seqlen/mean:29630.5 - actor/entropy:0.2007351815700531 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102352559566498 - training/rollout_probs_diff_mean:0.0033135332632809877 - training/rollout_probs_diff_std:0.01088031567633152 - training/rollout_actor_probs_pearson_corr:0.998772919178009 - rollout_corr/rollout_is_mean:1.0001227855682373 - rollout_corr/rollout_is_ratio_fraction_high:6.0338379626045935e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.033837780705653e-05 - rollout_corr/rollout_is_max:5.347679138183594 - rollout_corr/rollout_is_min:0.0047262562438845634 - rollout_corr/rollout_is_std:0.03222286328673363 - rollout_corr/rollout_is_eff_sample_size:0.998963001987778 - rollout_corr/rollout_is_seq_mean:1.00006902217865 - rollout_corr/rollout_is_seq_std:0.001625648234039545 - rollout_corr/rollout_is_seq_max:1.0073453187942505 - rollout_corr/rollout_is_seq_min:0.9958909749984741 - rollout_corr/rollout_is_seq_max_deviation:0.007345318794250488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.208143400028348) - rollout_corr/training_log_ppl:np.float64(0.17995586975303013) - rollout_corr/kl:np.float64(0.0005010489769254889) - rollout_corr/k3_kl:np.float64(0.0006270660993550337) - rollout_corr/rollout_ppl:np.float64(1.20752296410501) - rollout_corr/rollout_log_ppl:np.float64(0.1794548208345077) - rollout_corr/log_ppl_diff:np.float64(0.0005010489185224287) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013183121991460212) - rollout_corr/log_ppl_diff_max:np.float64(0.006140023469924927) - rollout_corr/log_ppl_diff_min:np.float64(-0.006050072610378265) - rollout_corr/ppl_ratio:np.float64(1.000502604758367) - rollout_corr/chi2_token:np.float64(0.0014851994346827269) - rollout_corr/chi2_seq:np.float64(1.3357745304238051) - actor/pg_loss:np.float64(-6.521574687212706e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033872173025883967) - actor/ppo_kl:np.float64(-1.556063463681312e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20266040414571762) - perf/mfu/actor:np.float64(0.08389738845778953) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0823974609375) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005806060507893562 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005806060507893562 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:647.390625 - response_length/max:2551.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:647.390625 - response_length_non_aborted/max:2551.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00031680427491664886 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9299852326512337) - timing_s/agent_loop/generate_sequences/max:np.float64(16.61781131848693) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.397257668249949) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.61781131848693) - timing_s/agent_loop/slowest
(TaskRunner pid=2049544) Training Progress: 45%|████▌ | 45/100 [32:51<39:37, 43.23s/it]
(TaskRunner pid=2049544) step:46 - global_seqlen/min:16300 - global_seqlen/max:40451 - global_seqlen/minmax_diff:24151 - global_seqlen/balanced_min:26008 - global_seqlen/balanced_max:26038 - global_seqlen/mean:26025.75 - actor/entropy:0.16465333104133606 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42921197414398193 - training/rollout_probs_diff_mean:0.0029608916956931353 - training/rollout_probs_diff_std:0.01070943009108305 - training/rollout_actor_probs_pearson_corr:0.9986132979393005 - rollout_corr/rollout_is_mean:0.9999878406524658 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.257039058255032e-05 - rollout_corr/rollout_is_max:1.9571115970611572 - rollout_corr/rollout_is_min:0.060181617736816406 - rollout_corr/rollout_is_std:0.030463820323348045 - rollout_corr/rollout_is_eff_sample_size:0.9990728161485534 - rollout_corr/rollout_is_seq_mean:1.000013828277588 - rollout_corr/rollout_is_seq_std:0.0019006438087671995 - rollout_corr/rollout_is_seq_max:1.008240818977356 - rollout_corr/rollout_is_seq_min:0.993937611579895 - rollout_corr/rollout_is_seq_max_deviation:0.008240818977355957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.153148335404694) - rollout_corr/training_log_ppl:np.float64(0.136645665843389) - rollout_corr/kl:np.float64(0.0006094530078737392) - rollout_corr/k3_kl:np.float64(0.0006117465548527434) - rollout_corr/rollout_ppl:np.float64(1.152437952812761) - rollout_corr/rollout_log_ppl:np.float64(0.13603621201764327) - rollout_corr/log_ppl_diff:np.float64(0.0006094538257457316) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014239892771001905) - rollout_corr/log_ppl_diff_max:np.float64(0.014275498688220978) - rollout_corr/log_ppl_diff_min:np.float64(-0.006409347057342529) - rollout_corr/ppl_ratio:np.float64(1.0006118323653936) - rollout_corr/chi2_token:np.float64(0.0012153126299381256) - rollout_corr/chi2_seq:np.float64(0.35479475394822657) - actor/pg_loss:np.float64(-5.846214480698109e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001732110890770855) - actor/ppo_kl:np.float64(5.357556297269639e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1466388632543385) - perf/mfu/actor:np.float64(0.07518990372656345) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.19413375854492) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023664643988013268 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023664643988013268 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:561.8671875 - response_length/max:2403.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:561.8671875 - response_length_non_aborted/max:2403.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002136826515197754 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7386901136487722) - timing_s/agent_loop/generate_sequences/max:np.float64(15.293935690075159) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.345510819919582) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.293935690075159) - timing_s/agent_loop/slowest
(TaskRunner pid=2049544) Training Progress: 46%|████▌ | 46/100 [33:25<36:26, 40.50s/it]
(TaskRunner pid=2049544) step:47 - global_seqlen/min:23716 - global_seqlen/max:42058 - global_seqlen/minmax_diff:18342 - global_seqlen/balanced_min:31374 - global_seqlen/balanced_max:31661 - global_seqlen/mean:31456.375 - actor/entropy:0.14881354570388794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4682484567165375 - training/rollout_probs_diff_mean:0.002767270663753152 - training/rollout_probs_diff_std:0.010147105902433395 - training/rollout_actor_probs_pearson_corr:0.9986124634742737 - rollout_corr/rollout_is_mean:1.0000232458114624 - rollout_corr/rollout_is_ratio_fraction_high:5.438477273855824e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.350781819084659e-05 - rollout_corr/rollout_is_max:2.263822555541992 - rollout_corr/rollout_is_min:0.43962690234184265 - rollout_corr/rollout_is_std:0.02890361100435257 - rollout_corr/rollout_is_eff_sample_size:0.999165278640662 - rollout_corr/rollout_is_seq_mean:1.0001194477081299 - rollout_corr/rollout_is_seq_std:0.0015050210058689117 - rollout_corr/rollout_is_seq_max:1.005627155303955 - rollout_corr/rollout_is_seq_min:0.993998646736145 - rollout_corr/rollout_is_seq_max_deviation:0.0060013532638549805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1467792866751552) - rollout_corr/training_log_ppl:np.float64(0.13201592379482463) - rollout_corr/kl:np.float64(0.0004759397659170883) - rollout_corr/k3_kl:np.float64(0.0005555009257420807) - rollout_corr/rollout_ppl:np.float64(1.146229465957731) - rollout_corr/rollout_log_ppl:np.float64(0.13153998326015426) - rollout_corr/log_ppl_diff:np.float64(0.00047594053467037156) - rollout_corr/log_ppl_abs_diff:np.float64(0.001261575052922126) - rollout_corr/log_ppl_diff_max:np.float64(0.008257180452346802) - rollout_corr/log_ppl_diff_min:np.float64(-0.0055009350180625916) - rollout_corr/ppl_ratio:np.float64(1.000477674184367) - rollout_corr/chi2_token:np.float64(0.0013230419717729092) - rollout_corr/chi2_seq:np.float64(0.636790500022471) - actor/pg_loss:np.float64(-0.00010656809899955988) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031795622612662555) - actor/ppo_kl:np.float64(0.00011273976376457995) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1950441598892212) - perf/mfu/actor:np.float64(0.0887384677499009) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.28681945800781) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011680489405989647 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011680489405989647 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:718.26171875 - response_length/max:4213.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:718.26171875 - response_length_non_aborted/max:4213.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017112679779529572 - timing_s/agent_loop/generate_sequences/min:np.float64(0.828915087506175) - timing_s/agent_loop/generate_sequences/max:np.float64(25.74352060444653) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.723139668560179) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.74352060444653) - timing_s/agent_l
(TaskRunner pid=2049544) Training Progress: 47%|████▋ | 47/100 [34:10<36:52, 41.75s/it]
(TaskRunner pid=2049544) step:48 - global_seqlen/min:18590 - global_seqlen/max:34823 - global_seqlen/minmax_diff:16233 - global_seqlen/balanced_min:28237 - global_seqlen/balanced_max:28801 - global_seqlen/mean:28407.875 - actor/entropy:0.14581364393234253 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47068536281585693 - training/rollout_probs_diff_mean:0.00291546992957592 - training/rollout_probs_diff_std:0.01052320096641779 - training/rollout_actor_probs_pearson_corr:0.9985108971595764 - rollout_corr/rollout_is_mean:0.9999670386314392 - rollout_corr/rollout_is_ratio_fraction_high:1.2597076420206577e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.779122926061973e-05 - rollout_corr/rollout_is_max:2.1663451194763184 - rollout_corr/rollout_is_min:0.35977837443351746 - rollout_corr/rollout_is_std:0.030446205288171768 - rollout_corr/rollout_is_eff_sample_size:0.9990738870447088 - rollout_corr/rollout_is_seq_mean:0.9999531507492065 - rollout_corr/rollout_is_seq_std:0.0016499547054991126 - rollout_corr/rollout_is_seq_max:1.0073652267456055 - rollout_corr/rollout_is_seq_min:0.9916918873786926 - rollout_corr/rollout_is_seq_max_deviation:0.008308112621307373 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1378675876185298) - rollout_corr/training_log_ppl:np.float64(0.12675013709667837) - rollout_corr/kl:np.float64(0.000753273961016987) - rollout_corr/k3_kl:np.float64(0.0006820533353391056) - rollout_corr/rollout_ppl:np.float64(1.1369990683160722) - rollout_corr/rollout_log_ppl:np.float64(0.12599686199246207) - rollout_corr/log_ppl_diff:np.float64(0.0007532751042163) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014732455310877413) - rollout_corr/log_ppl_diff_max:np.float64(0.009199053049087524) - rollout_corr/log_ppl_diff_min:np.float64(-0.006944321095943451) - rollout_corr/ppl_ratio:np.float64(1.0007553710602224) - rollout_corr/chi2_token:np.float64(0.0013299805577844381) - rollout_corr/chi2_seq:np.float64(0.6524860400240868) - actor/pg_loss:np.float64(-2.4408800527453423e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007820084601917188) - actor/ppo_kl:np.float64(0.00018927762639009416) - actor/pg_clipfrac_lower:np.float64(1.1097301467088982e-05) - actor/grad_norm:np.float64(0.2395218200981617) - perf/mfu/actor:np.float64(0.08095804530180152) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.21407699584961) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01809648796916008 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01809648796916008 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:620.18359375 - response_length/max:3666.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:620.18359375 - response_length_non_aborted/max:3666.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014730170369148254 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9394192546606064) - timing_s/agent_loop/generate_sequences/max:np.float64(21.90096348337829) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.9833613064474775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.900
(TaskRunner pid=2049544) Training Progress: 48%|████▊ | 48/100 [34:51<36:03, 41.61s/it]
(TaskRunner pid=2049544) step:49 - global_seqlen/min:18126 - global_seqlen/max:38899 - global_seqlen/minmax_diff:20773 - global_seqlen/balanced_min:26338 - global_seqlen/balanced_max:26710 - global_seqlen/mean:26451.75 - actor/entropy:0.18256059288978577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3656392991542816 - training/rollout_probs_diff_mean:0.0032566494774073362 - training/rollout_probs_diff_std:0.010963589884340763 - training/rollout_actor_probs_pearson_corr:0.9987143278121948 - rollout_corr/rollout_is_mean:1.0001035928726196 - rollout_corr/rollout_is_ratio_fraction_high:6.82920153849409e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.146281521068886e-05 - rollout_corr/rollout_is_max:2.4023168087005615 - rollout_corr/rollout_is_min:0.14749553799629211 - rollout_corr/rollout_is_std:0.03197216987609863 - rollout_corr/rollout_is_eff_sample_size:0.9989789431740315 - rollout_corr/rollout_is_seq_mean:1.0002081394195557 - rollout_corr/rollout_is_seq_std:0.001892627333290875 - rollout_corr/rollout_is_seq_max:1.008901834487915 - rollout_corr/rollout_is_seq_min:0.9917373061180115 - rollout_corr/rollout_is_seq_max_deviation:0.008901834487915039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1714685205370188) - rollout_corr/training_log_ppl:np.float64(0.15115834452444687) - rollout_corr/kl:np.float64(0.0004743365148804912) - rollout_corr/k3_kl:np.float64(0.0006857500739982925) - rollout_corr/rollout_ppl:np.float64(1.1708842911757529) - rollout_corr/rollout_log_ppl:np.float64(0.15068400763266254) - rollout_corr/log_ppl_diff:np.float64(0.0004743368917843327) - rollout_corr/log_ppl_abs_diff:np.float64(0.001488212525146082) - rollout_corr/log_ppl_diff_max:np.float64(0.010617084801197052) - rollout_corr/log_ppl_diff_min:np.float64(-0.015819266438484192) - rollout_corr/ppl_ratio:np.float64(1.0004769298247993) - rollout_corr/chi2_token:np.float64(0.0018537091091275215) - rollout_corr/chi2_seq:np.float64(0.6309683343861252) - actor/pg_loss:np.float64(3.3562071621418e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005490989437930693) - actor/ppo_kl:np.float64(8.648514501530258e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2604895681142807) - perf/mfu/actor:np.float64(0.07564944214215971) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.27551984786987) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015421191230416298 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015421191230416298 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:571.9921875 - response_length/max:3613.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.9921875 - response_length_non_aborted/max:3613.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014311634004116058 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9619488045573235) - timing_s/agent_loop/generate_sequences/max:np.float64(21.043767675757408) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.410715061574592) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.043767675757408) - timing_s/a
(TaskRunner pid=2049544) Training Progress: 49%|████▉ | 49/100 [35:31<35:01, 41.20s/it]
(TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50}
(TaskRunner pid=2049544) validation generation end
(TaskRunner pid=2049544) [prompt] system
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2049544) <reasoning>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) A
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) user
(TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) A: 50 pC
(TaskRunner pid=2049544) B: 56 pC
(TaskRunner pid=2049544) C: 60 pC
(TaskRunner pid=2049544) D: 64 pC
(TaskRunner pid=2049544) Please reason step by step.
(TaskRunner pid=2049544) assistant
(TaskRunner pid=2049544) <think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) [response] <reasoning>
(TaskRunner pid=2049544) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) E = k \frac{q}{r^2}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) We are given:
(TaskRunner pid=2049544) - $ E = 2.0 \, \mathrm{N/C} $
(TaskRunner pid=2049544) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $
(TaskRunner pid=2049544) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) We need to solve for $ q $:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{E r^2}{k}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Substitute the values:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Convert to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $):
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) The closest option is **B: 56 pC**.
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) B
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544) [ground_truth] B
(TaskRunner pid=2049544) [score] 1.0
(TaskRunner pid=2049544) [acc] 1.0
(TaskRunner pid=2049544) [pred] B
(TaskRunner pid=2049544) [incorrect_format] 1
(TaskRunner pid=2049544) [feedback]
(TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50
(WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor
(TaskRunner pid=2049544) step:50 - global_seqlen/min:22621 - global_seqlen/max:38182 - global_seqlen/minmax_diff:15561 - global_seqlen/balanced_min:31307 - global_seqlen/balanced_max:32242 - global_seqlen/mean:31432.0 - actor/entropy:0.20185792446136475 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3018432855606079 - training/rollout_probs_diff_mean:0.003106999211013317 - training/rollout_probs_diff_std:0.010115313343703747 - training/rollout_actor_probs_pearson_corr:0.9989494681358337 - rollout_corr/rollout_is_mean:1.0000050067901611 - rollout_corr/rollout_is_ratio_fraction_high:5.458515261125285e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.458515261125285e-06 - rollout_corr/rollout_is_max:2.0969016551971436 - rollout_corr/rollout_is_min:0.33815088868141174 - rollout_corr/rollout_is_std:0.030253741890192032 - rollout_corr/rollout_is_eff_sample_size:0.9990853100798829 - rollout_corr/rollout_is_seq_mean:0.9999303817749023 - rollout_corr/rollout_is_seq_std:0.001567038125358522 - rollout_corr/rollout_is_seq_max:1.0050349235534668 - rollout_corr/rollout_is_seq_min:0.993672788143158 - rollout_corr/rollout_is_seq_max_deviation:0.006327211856842041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.191426008939743) - rollout_corr/training_log_ppl:np.float64(0.16859979891887633) - rollout_corr/kl:np.float64(0.0005692955341736194) - rollout_corr/k3_kl:np.float64(0.0005476311655883137) - rollout_corr/rollout_ppl:np.float64(1.1907334956340492) - rollout_corr/rollout_log_ppl:np.float64(0.16803050318412716) - rollout_corr/log_ppl_diff:np.float64(0.0005692957347491756) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012659797939704731) - rollout_corr/log_ppl_diff_max:np.float64(0.008102566003799438) - rollout_corr/log_ppl_diff_min:np.float64(-0.00437454879283905) - rollout_corr/ppl_ratio:np.float64(1.0005708439275622) - rollout_corr/chi2_token:np.float64(0.0010575931519269943) - rollout_corr/chi2_seq:np.float64(1.1911898551043123) - actor/pg_loss:np.float64(6.677815690636635e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036017004458699375) - actor/ppo_kl:np.float64(2.6112116451315615e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17873636819422245) - perf/mfu/actor:np.float64(0.0885810936633662) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.26837921142578) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09966534252010995) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8278625) - val-aux/sciknow
(TaskRunner pid=2049544) Training Progress: 50%|█████ | 50/100 [38:18<1:05:48, 78.97s/it]
(TaskRunner pid=2049544) step:51 - global_seqlen/min:18785 - global_seqlen/max:33808 - global_seqlen/minmax_diff:15023 - global_seqlen/balanced_min:26399 - global_seqlen/balanced_max:26471 - global_seqlen/mean:26450.125 - actor/entropy:0.24430422484874725 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564788341522217 - training/rollout_probs_diff_mean:0.003582285949960351 - training/rollout_probs_diff_std:0.010883934795856476 - training/rollout_actor_probs_pearson_corr:0.999004602432251 - rollout_corr/rollout_is_mean:0.999866783618927 - rollout_corr/rollout_is_ratio_fraction_high:1.396638253936544e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.189914761809632e-05 - rollout_corr/rollout_is_max:2.2966716289520264 - rollout_corr/rollout_is_min:0.327895849943161 - rollout_corr/rollout_is_std:0.033104244619607925 - rollout_corr/rollout_is_eff_sample_size:0.9989051897151183 - rollout_corr/rollout_is_seq_mean:0.9998487830162048 - rollout_corr/rollout_is_seq_std:0.0018385471776127815 - rollout_corr/rollout_is_seq_max:1.0075657367706299 - rollout_corr/rollout_is_seq_min:0.9936858415603638 - rollout_corr/rollout_is_seq_max_deviation:0.007565736770629883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.222205103840679) - rollout_corr/training_log_ppl:np.float64(0.18761249752424192) - rollout_corr/kl:np.float64(0.0007189881607398974) - rollout_corr/k3_kl:np.float64(0.0005682342473676272) - rollout_corr/rollout_ppl:np.float64(1.2213212260976434) - rollout_corr/rollout_log_ppl:np.float64(0.1868935083039105) - rollout_corr/log_ppl_diff:np.float64(0.000718989220331423) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014408860151888803) - rollout_corr/log_ppl_diff_max:np.float64(0.007139421999454498) - rollout_corr/log_ppl_diff_min:np.float64(-0.005957402288913727) - rollout_corr/ppl_ratio:np.float64(1.0007210047915578) - rollout_corr/chi2_token:np.float64(0.0008357972837984562) - rollout_corr/chi2_seq:np.float64(1.0313024234492332) - actor/pg_loss:np.float64(4.522060044109821e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003210717095498694) - actor/ppo_kl:np.float64(-9.128155866733323e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14451957494020462) - perf/mfu/actor:np.float64(0.07523053933355461) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.30613327026367) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011186200194060802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011186200194060802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:559.37890625 - response_length/max:2702.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:559.37890625 - response_length_non_aborted/max:2702.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.59448516368866e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9070874620229006) - timing_s/agent_loop/generate_sequences/max:np.float64(16.53190640732646) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.214220573740022) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.53190640732646) - timing_
(TaskRunner pid=2049544) Training Progress: 51%|█████ | 51/100 [38:54<53:58, 66.09s/it]
(TaskRunner pid=2049544) step:52 - global_seqlen/min:23429 - global_seqlen/max:36160 - global_seqlen/minmax_diff:12731 - global_seqlen/balanced_min:30668 - global_seqlen/balanced_max:30765 - global_seqlen/mean:30739.125 - actor/entropy:0.22692662477493286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3938141167163849 - training/rollout_probs_diff_mean:0.0033940812572836876 - training/rollout_probs_diff_std:0.010467931628227234 - training/rollout_actor_probs_pearson_corr:0.9990243315696716 - rollout_corr/rollout_is_mean:1.000043272972107 - rollout_corr/rollout_is_ratio_fraction_high:1.1359441487002186e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2718882974004373e-05 - rollout_corr/rollout_is_max:2.0554285049438477 - rollout_corr/rollout_is_min:0.29720351099967957 - rollout_corr/rollout_is_std:0.03246244043111801 - rollout_corr/rollout_is_eff_sample_size:0.9989472992270986 - rollout_corr/rollout_is_seq_mean:1.0000196695327759 - rollout_corr/rollout_is_seq_std:0.001915202010422945 - rollout_corr/rollout_is_seq_max:1.0059826374053955 - rollout_corr/rollout_is_seq_min:0.9903362989425659 - rollout_corr/rollout_is_seq_max_deviation:0.009663701057434082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.231195269152522) - rollout_corr/training_log_ppl:np.float64(0.19834982651809696) - rollout_corr/kl:np.float64(0.0008127379676388102) - rollout_corr/k3_kl:np.float64(0.0006191496557619303) - rollout_corr/rollout_ppl:np.float64(1.230196955613792) - rollout_corr/rollout_log_ppl:np.float64(0.19753708717325935) - rollout_corr/log_ppl_diff:np.float64(0.0008127393448376097) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014782494035898708) - rollout_corr/log_ppl_diff_max:np.float64(0.011166423559188843) - rollout_corr/log_ppl_diff_min:np.float64(-0.00570167601108551) - rollout_corr/ppl_ratio:np.float64(1.0008151046931744) - rollout_corr/chi2_token:np.float64(0.0008716660086065531) - rollout_corr/chi2_seq:np.float64(0.30706389155238867) - actor/pg_loss:np.float64(2.5613000616431236e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015267897435933264) - actor/ppo_kl:np.float64(0.00024156836637168055) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18445459008216858) - perf/mfu/actor:np.float64(0.08808419007285372) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.37739562988281) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007727969903498888 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007727969903498888 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:687.75390625 - response_length/max:2764.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:687.75390625 - response_length_non_aborted/max:2764.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011474080383777618 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7448907978832722) - timing_s/agent_loop/generate_sequences/max:np.float64(18.379133939743042) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.580681977618951) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.379133939743042) - timin
(TaskRunner pid=2049544) Training Progress: 52%|█████▏ | 52/100 [39:32<45:57, 57.45s/it]
(TaskRunner pid=2049544) step:53 - global_seqlen/min:19572 - global_seqlen/max:34404 - global_seqlen/minmax_diff:14832 - global_seqlen/balanced_min:27920 - global_seqlen/balanced_max:29013 - global_seqlen/mean:28188.25 - actor/entropy:0.22438420355319977 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30675768852233887 - training/rollout_probs_diff_mean:0.0035303884651511908 - training/rollout_probs_diff_std:0.010939665138721466 - training/rollout_actor_probs_pearson_corr:0.9988002181053162 - rollout_corr/rollout_is_mean:1.000083327293396 - rollout_corr/rollout_is_ratio_fraction_high:1.938861169037409e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.524009636952542e-05 - rollout_corr/rollout_is_max:2.4050400257110596 - rollout_corr/rollout_is_min:0.11861640214920044 - rollout_corr/rollout_is_std:0.03258340060710907 - rollout_corr/rollout_is_eff_sample_size:0.9989399238583818 - rollout_corr/rollout_is_seq_mean:1.0001201629638672 - rollout_corr/rollout_is_seq_std:0.001550388871692121 - rollout_corr/rollout_is_seq_max:1.0046480894088745 - rollout_corr/rollout_is_seq_min:0.994028627872467 - rollout_corr/rollout_is_seq_max_deviation:0.005971372127532959 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2066223253495991) - rollout_corr/training_log_ppl:np.float64(0.17772597989824135) - rollout_corr/kl:np.float64(0.0005446721567352597) - rollout_corr/k3_kl:np.float64(0.0006030060052921726) - rollout_corr/rollout_ppl:np.float64(1.2059485423378646) - rollout_corr/rollout_log_ppl:np.float64(0.17718130597495474) - rollout_corr/log_ppl_diff:np.float64(0.0005446739232866094) - rollout_corr/log_ppl_abs_diff:np.float64(0.001306041274801828) - rollout_corr/log_ppl_diff_max:np.float64(0.006498947739601135) - rollout_corr/log_ppl_diff_min:np.float64(-0.005862616002559662) - rollout_corr/ppl_ratio:np.float64(1.0005462081171572) - rollout_corr/chi2_token:np.float64(0.001310169231146574) - rollout_corr/chi2_seq:np.float64(0.6170898247510195) - actor/pg_loss:np.float64(5.588738713413477e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040137838948339777) - actor/ppo_kl:np.float64(0.00010251342614253645) - actor/pg_clipfrac_lower:np.float64(1.1289739632047713e-06) - actor/grad_norm:np.float64(0.2714943513274193) - perf/mfu/actor:np.float64(0.0789630435605575) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.31622695922852) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02330026589334011 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02330026589334011 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:604.4140625 - response_length/max:3739.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:604.4140625 - response_length_non_aborted/max:3739.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001577865332365036 - timing_s/agent_loop/generate_sequences/min:np.float64(1.263271963223815) - timing_s/agent_loop/generate_sequences/max:np.float64(22.0587509367615) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.729596399491129) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.058750936
(TaskRunner pid=2049544) Training Progress: 53%|█████▎ | 53/100 [40:13<41:15, 52.67s/it]
(TaskRunner pid=2049544) step:54 - global_seqlen/min:21471 - global_seqlen/max:31367 - global_seqlen/minmax_diff:9896 - global_seqlen/balanced_min:26297 - global_seqlen/balanced_max:26328 - global_seqlen/mean:26318.0 - actor/entropy:0.21818441152572632 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292486190795898 - training/rollout_probs_diff_mean:0.0032305221538990736 - training/rollout_probs_diff_std:0.010491388849914074 - training/rollout_actor_probs_pearson_corr:0.9989482760429382 - rollout_corr/rollout_is_mean:1.0000078678131104 - rollout_corr/rollout_is_ratio_fraction_high:2.0765270164702088e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3843512533640023e-05 - rollout_corr/rollout_is_max:2.25616192817688 - rollout_corr/rollout_is_min:0.26139989495277405 - rollout_corr/rollout_is_std:0.03054003044962883 - rollout_corr/rollout_is_eff_sample_size:0.999068175625075 - rollout_corr/rollout_is_seq_mean:0.9999727010726929 - rollout_corr/rollout_is_seq_std:0.001957092434167862 - rollout_corr/rollout_is_seq_max:1.007343053817749 - rollout_corr/rollout_is_seq_min:0.9894382953643799 - rollout_corr/rollout_is_seq_max_deviation:0.010561704635620117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1926028821617365) - rollout_corr/training_log_ppl:np.float64(0.16857526991225313) - rollout_corr/kl:np.float64(0.0005841684274781045) - rollout_corr/k3_kl:np.float64(0.0005794948536674838) - rollout_corr/rollout_ppl:np.float64(1.1918872729875147) - rollout_corr/rollout_log_ppl:np.float64(0.1679911007740884) - rollout_corr/log_ppl_diff:np.float64(0.0005841691381647252) - rollout_corr/log_ppl_abs_diff:np.float64(0.001538367178000044) - rollout_corr/log_ppl_diff_max:np.float64(0.011501207947731018) - rollout_corr/log_ppl_diff_min:np.float64(-0.005205966532230377) - rollout_corr/ppl_ratio:np.float64(1.0005866582505405) - rollout_corr/chi2_token:np.float64(0.0011341231875121593) - rollout_corr/chi2_seq:np.float64(1.4627508698031306) - actor/pg_loss:np.float64(0.00015633960720151663) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00042431396536812827) - actor/ppo_kl:np.float64(3.235917171728886e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16759618930518627) - perf/mfu/actor:np.float64(0.07638956882275981) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.35180282592773) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008146907202899456 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008146907202899456 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:564.34375 - response_length/max:2093.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:564.34375 - response_length_non_aborted/max:2093.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015513785183429718 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6626291759312153) - timing_s/agent_loop/generate_sequences/max:np.float64(14.199840543791652) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.107884573328192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.199840543791652) - timing_s/a
(TaskRunner pid=2049544) Training Progress: 54%|█████▍ | 54/100 [40:46<35:52, 46.80s/it]
(TaskRunner pid=2049544) step:55 - global_seqlen/min:18191 - global_seqlen/max:33987 - global_seqlen/minmax_diff:15796 - global_seqlen/balanced_min:26610 - global_seqlen/balanced_max:26684 - global_seqlen/mean:26660.125 - actor/entropy:0.1736723929643631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105892539024353 - training/rollout_probs_diff_mean:0.0030401991680264473 - training/rollout_probs_diff_std:0.01039278693497181 - training/rollout_actor_probs_pearson_corr:0.9987287521362305 - rollout_corr/rollout_is_mean:1.0000579357147217 - rollout_corr/rollout_is_ratio_fraction_high:1.375430656480603e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.375430656480603e-05 - rollout_corr/rollout_is_max:2.335618734359741 - rollout_corr/rollout_is_min:0.392137348651886 - rollout_corr/rollout_is_std:0.030324585735797882 - rollout_corr/rollout_is_eff_sample_size:0.999081502372468 - rollout_corr/rollout_is_seq_mean:1.000092625617981 - rollout_corr/rollout_is_seq_std:0.00183792260941118 - rollout_corr/rollout_is_seq_max:1.0104801654815674 - rollout_corr/rollout_is_seq_min:0.9918577671051025 - rollout_corr/rollout_is_seq_max_deviation:0.010480165481567383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1669259667396545) - rollout_corr/training_log_ppl:np.float64(0.14700542857826804) - rollout_corr/kl:np.float64(0.0007857797350844242) - rollout_corr/k3_kl:np.float64(0.0007165672559068526) - rollout_corr/rollout_ppl:np.float64(1.1659750957041979) - rollout_corr/rollout_log_ppl:np.float64(0.14621964823891176) - rollout_corr/log_ppl_diff:np.float64(0.000785780339356279) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016149269104062114) - rollout_corr/log_ppl_diff_max:np.float64(0.058644767850637436) - rollout_corr/log_ppl_diff_min:np.float64(-0.004740942269563675) - rollout_corr/ppl_ratio:np.float64(1.00079559488222) - rollout_corr/chi2_token:np.float64(0.0010453276336193085) - rollout_corr/chi2_seq:np.float64(0.6579223412554711) - actor/pg_loss:np.float64(-0.00018631992861628532) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005601525168685839) - actor/ppo_kl:np.float64(0.00037079197026113064) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25370683521032333) - perf/mfu/actor:np.float64(0.0767427855529022) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.34751892089844) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002467178856022656 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002467178856022656 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:568.00390625 - response_length/max:2618.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:568.00390625 - response_length_non_aborted/max:2618.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001354403793811798 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8230457752943039) - timing_s/agent_loop/generate_sequences/max:np.float64(16.547815980389714) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.079917931871023) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.547815980389714) - timing_
(TaskRunner pid=2049544) Training Progress: 55%|█████▌ | 55/100 [41:22<32:37, 43.51s/it]
(TaskRunner pid=2049544) step:56 - global_seqlen/min:22164 - global_seqlen/max:48288 - global_seqlen/minmax_diff:26124 - global_seqlen/balanced_min:28371 - global_seqlen/balanced_max:28431 - global_seqlen/mean:28409.375 - actor/entropy:0.21692201495170593 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3174453377723694 - training/rollout_probs_diff_mean:0.003212681272998452 - training/rollout_probs_diff_std:0.010248677805066109 - training/rollout_actor_probs_pearson_corr:0.9989055395126343 - rollout_corr/rollout_is_mean:1.0000497102737427 - rollout_corr/rollout_is_ratio_fraction_high:1.2312617400311865e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.309416181058623e-05 - rollout_corr/rollout_is_max:2.551107883453369 - rollout_corr/rollout_is_min:0.3708029091358185 - rollout_corr/rollout_is_std:0.031356628984212875 - rollout_corr/rollout_is_eff_sample_size:0.9990177275875637 - rollout_corr/rollout_is_seq_mean:1.0000814199447632 - rollout_corr/rollout_is_seq_std:0.002115494105964899 - rollout_corr/rollout_is_seq_max:1.0103763341903687 - rollout_corr/rollout_is_seq_min:0.9925733208656311 - rollout_corr/rollout_is_seq_max_deviation:0.010376334190368652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2074726885184646) - rollout_corr/training_log_ppl:np.float64(0.18105585960438475) - rollout_corr/kl:np.float64(0.0007089410478542391) - rollout_corr/k3_kl:np.float64(0.0007255429266592728) - rollout_corr/rollout_ppl:np.float64(1.2066095937043428) - rollout_corr/rollout_log_ppl:np.float64(0.18034691788489) - rollout_corr/log_ppl_diff:np.float64(0.00070894171949476) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016144337132573128) - rollout_corr/log_ppl_diff_max:np.float64(0.014080479741096497) - rollout_corr/log_ppl_diff_min:np.float64(-0.00835387408733368) - rollout_corr/ppl_ratio:np.float64(1.0007117404602468) - rollout_corr/chi2_token:np.float64(0.0016396085266023874) - rollout_corr/chi2_seq:np.float64(0.7452940770890564) - actor/pg_loss:np.float64(0.00023605907335877419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041431330055274884) - actor/ppo_kl:np.float64(0.00019277200499256963) - actor/pg_clipfrac_lower:np.float64(1.5439723938470706e-05) - actor/grad_norm:np.float64(0.27160290628671646) - perf/mfu/actor:np.float64(0.08149403640203684) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.39385986328125) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020954536739736795 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020954536739736795 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:634.51171875 - response_length/max:2697.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:634.51171875 - response_length_non_aborted/max:2697.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.834898471832275e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9801836125552654) - timing_s/agent_loop/generate_sequences/max:np.float64(17.489308523014188) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.853927971802477) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.4893
(TaskRunner pid=2049544) Training Progress: 56%|█████▌ | 56/100 [41:59<30:19, 41.35s/it]
(TaskRunner pid=2049544) step:57 - global_seqlen/min:25396 - global_seqlen/max:39989 - global_seqlen/minmax_diff:14593 - global_seqlen/balanced_min:32645 - global_seqlen/balanced_max:32684 - global_seqlen/mean:32673.25 - actor/entropy:0.20889751613140106 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23668746650218964 - training/rollout_probs_diff_mean:0.0030729498248547316 - training/rollout_probs_diff_std:0.009970693849027157 - training/rollout_actor_probs_pearson_corr:0.9990144371986389 - rollout_corr/rollout_is_mean:0.9999489188194275 - rollout_corr/rollout_is_ratio_fraction_high:5.2733157644979656e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.691321035148576e-05 - rollout_corr/rollout_is_max:2.685802698135376 - rollout_corr/rollout_is_min:0.4615940749645233 - rollout_corr/rollout_is_std:0.03016890585422516 - rollout_corr/rollout_is_eff_sample_size:0.9990906647175409 - rollout_corr/rollout_is_seq_mean:0.9998156428337097 - rollout_corr/rollout_is_seq_std:0.0015230864519253373 - rollout_corr/rollout_is_seq_max:1.0045884847640991 - rollout_corr/rollout_is_seq_min:0.9938079118728638 - rollout_corr/rollout_is_seq_max_deviation:0.0061920881271362305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2169137438759208) - rollout_corr/training_log_ppl:np.float64(0.18850286441738717) - rollout_corr/kl:np.float64(0.0008272747242870082) - rollout_corr/k3_kl:np.float64(0.0005567002334601057) - rollout_corr/rollout_ppl:np.float64(1.2158889258280396) - rollout_corr/rollout_log_ppl:np.float64(0.18767558728723088) - rollout_corr/log_ppl_diff:np.float64(0.0008272771301562898) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012787482482963242) - rollout_corr/log_ppl_diff_max:np.float64(0.00949886441230774) - rollout_corr/log_ppl_diff_min:np.float64(-0.003404974937438965) - rollout_corr/ppl_ratio:np.float64(1.000829064520076) - rollout_corr/chi2_token:np.float64(0.0005316368769854307) - rollout_corr/chi2_seq:np.float64(0.33462699479423463) - actor/pg_loss:np.float64(4.014407750219107e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038230796246807586) - actor/ppo_kl:np.float64(0.00015057841544852124) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21619294956326485) - perf/mfu/actor:np.float64(0.09399730782746046) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.41035079956055) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012718578800559044 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012718578800559044 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:740.7578125 - response_length/max:2402.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:740.7578125 - response_length_non_aborted/max:2402.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010364875197410583 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0282678361982107) - timing_s/agent_loop/generate_sequences/max:np.float64(16.842207921668887) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.293985813957988) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.842207921668887)
(TaskRunner pid=2049544) Training Progress: 57%|█████▋ | 57/100 [42:34<28:26, 39.69s/it]
(TaskRunner pid=2049544) step:58 - global_seqlen/min:19068 - global_seqlen/max:41428 - global_seqlen/minmax_diff:22360 - global_seqlen/balanced_min:32655 - global_seqlen/balanced_max:33260 - global_seqlen/mean:32761.875 - actor/entropy:0.25287628173828125 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42262715101242065 - training/rollout_probs_diff_mean:0.0032485839910805225 - training/rollout_probs_diff_std:0.009784280322492123 - training/rollout_actor_probs_pearson_corr:0.9992220997810364 - rollout_corr/rollout_is_mean:0.9999250769615173 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.117093729088083e-05 - rollout_corr/rollout_is_max:1.704056739807129 - rollout_corr/rollout_is_min:0.30506160855293274 - rollout_corr/rollout_is_std:0.030359944328665733 - rollout_corr/rollout_is_eff_sample_size:0.9990791225700723 - rollout_corr/rollout_is_seq_mean:0.9998817443847656 - rollout_corr/rollout_is_seq_std:0.0017060940153896809 - rollout_corr/rollout_is_seq_max:1.0051213502883911 - rollout_corr/rollout_is_seq_min:0.9917200803756714 - rollout_corr/rollout_is_seq_max_deviation:0.008279919624328613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.248608773574233) - rollout_corr/training_log_ppl:np.float64(0.2106571579060983) - rollout_corr/kl:np.float64(0.0006302413064700474) - rollout_corr/k3_kl:np.float64(0.0005439822076880318) - rollout_corr/rollout_ppl:np.float64(1.2478208504617214) - rollout_corr/rollout_log_ppl:np.float64(0.21002691524336115) - rollout_corr/log_ppl_diff:np.float64(0.000630242662737146) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013482350041158497) - rollout_corr/log_ppl_diff_max:np.float64(0.007139943540096283) - rollout_corr/log_ppl_diff_min:np.float64(-0.009232334792613983) - rollout_corr/ppl_ratio:np.float64(1.0006320539396256) - rollout_corr/chi2_token:np.float64(0.0009030492510646582) - rollout_corr/chi2_seq:np.float64(2.417929224204272) - actor/pg_loss:np.float64(-9.389116894453764e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003193014416638107) - actor/ppo_kl:np.float64(-1.695539558355108e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21667541190981865) - perf/mfu/actor:np.float64(0.09071205098155613) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.45818328857422) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013659363612532616 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013659363612532616 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:751.90234375 - response_length/max:4193.0 - response_length/min:62.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:751.90234375 - response_length_non_aborted/max:4193.0 - response_length_non_aborted/min:62.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.30633395910263e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7540517617017031) - timing_s/agent_loop/generate_sequences/max:np.float64(25.78281068429351) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.984791398346715) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.78281068429351) - timing_s/agent_loop/slo
(TaskRunner pid=2049544) Training Progress: 58%|█████▊ | 58/100 [43:19<28:54, 41.29s/it]
(TaskRunner pid=2049544) step:59 - global_seqlen/min:23178 - global_seqlen/max:36252 - global_seqlen/minmax_diff:13074 - global_seqlen/balanced_min:28566 - global_seqlen/balanced_max:28612 - global_seqlen/mean:28596.625 - actor/entropy:0.24240879714488983 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797503709793091 - training/rollout_probs_diff_mean:0.0033322961535304785 - training/rollout_probs_diff_std:0.010173420421779156 - training/rollout_actor_probs_pearson_corr:0.9990172982215881 - rollout_corr/rollout_is_mean:1.0001167058944702 - rollout_corr/rollout_is_ratio_fraction_high:6.2438730310532264e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.873162000265438e-05 - rollout_corr/rollout_is_max:2.413045644760132 - rollout_corr/rollout_is_min:0.38240286707878113 - rollout_corr/rollout_is_std:0.03103756159543991 - rollout_corr/rollout_is_eff_sample_size:0.9990379537831949 - rollout_corr/rollout_is_seq_mean:1.0002449750900269 - rollout_corr/rollout_is_seq_std:0.0013697257963940501 - rollout_corr/rollout_is_seq_max:1.003889560699463 - rollout_corr/rollout_is_seq_min:0.9955812692642212 - rollout_corr/rollout_is_seq_max_deviation:0.004418730735778809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2532247374765575) - rollout_corr/training_log_ppl:np.float64(0.21026221416104818) - rollout_corr/kl:np.float64(0.00036410490782223803) - rollout_corr/k3_kl:np.float64(0.0005213920420086282) - rollout_corr/rollout_ppl:np.float64(1.2527254964224994) - rollout_corr/rollout_log_ppl:np.float64(0.20989810848550405) - rollout_corr/log_ppl_diff:np.float64(0.0003641056755441241) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011843454485642724) - rollout_corr/log_ppl_diff_max:np.float64(0.0051978230476379395) - rollout_corr/log_ppl_diff_min:np.float64(-0.004664614796638489) - rollout_corr/ppl_ratio:np.float64(1.000365313142538) - rollout_corr/chi2_token:np.float64(0.0013532442972064018) - rollout_corr/chi2_seq:np.float64(1.167969731381163) - actor/pg_loss:np.float64(5.0825648941099644e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027953678352332645) - actor/ppo_kl:np.float64(0.0001204940012513589) - actor/pg_clipfrac_lower:np.float64(2.0624340777430916e-06) - actor/grad_norm:np.float64(0.2315312847495079) - perf/mfu/actor:np.float64(0.08222550993023878) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.56365966796875) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007726792828179896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007726792828179896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:625.61328125 - response_length/max:2175.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:625.61328125 - response_length_non_aborted/max:2175.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.861107587814331e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0195303186774254) - timing_s/agent_loop/generate_sequences/max:np.float64(15.18119196780026) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.312842677085428) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(1
(TaskRunner pid=2049544) Training Progress: 59%|█████▉ | 59/100 [43:54<26:48, 39.22s/it]
(TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60}
(TaskRunner pid=2049544) validation generation end
(TaskRunner pid=2049544) [prompt] system
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2049544) <reasoning>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) A
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) user
(TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) A: 50 pC
(TaskRunner pid=2049544) B: 56 pC
(TaskRunner pid=2049544) C: 60 pC
(TaskRunner pid=2049544) D: 64 pC
(TaskRunner pid=2049544) Please reason step by step.
(TaskRunner pid=2049544) assistant
(TaskRunner pid=2049544) <think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) [response] <reasoning>
(TaskRunner pid=2049544) We are given:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $
(TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $
(TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) We are to find the charge $ q $ using the formula:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) E = k \frac{q}{r^2}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Rearranging for $ q $:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Substitute the known values:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{0.5}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $):
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) The closest option is **B: 56 pC**.
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) B
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544) [ground_truth] B
(TaskRunner pid=2049544) [score] 1.0
(TaskRunner pid=2049544) [acc] 1.0
(TaskRunner pid=2049544) [pred] B
(TaskRunner pid=2049544) [incorrect_format] 1
(TaskRunner pid=2049544) [feedback]
(TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60
(WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor
(TaskRunner pid=2049544) step:60 - global_seqlen/min:19702 - global_seqlen/max:41022 - global_seqlen/minmax_diff:21320 - global_seqlen/balanced_min:29482 - global_seqlen/balanced_max:29782 - global_seqlen/mean:29534.0 - actor/entropy:0.2471179962158203 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2885955274105072 - training/rollout_probs_diff_mean:0.0032362197525799274 - training/rollout_probs_diff_std:0.009918652474880219 - training/rollout_actor_probs_pearson_corr:0.9991265535354614 - rollout_corr/rollout_is_mean:1.0000226497650146 - rollout_corr/rollout_is_ratio_fraction_high:1.1968021681241225e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1968021681241225e-05 - rollout_corr/rollout_is_max:2.9771621227264404 - rollout_corr/rollout_is_min:0.3293769657611847 - rollout_corr/rollout_is_std:0.03091634064912796 - rollout_corr/rollout_is_eff_sample_size:0.999045211617763 - rollout_corr/rollout_is_seq_mean:0.9999569654464722 - rollout_corr/rollout_is_seq_std:0.0014956871746107936 - rollout_corr/rollout_is_seq_max:1.0054192543029785 - rollout_corr/rollout_is_seq_min:0.9950917959213257 - rollout_corr/rollout_is_seq_max_deviation:0.005419254302978516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2401416734792292) - rollout_corr/training_log_ppl:np.float64(0.19959818758798065) - rollout_corr/kl:np.float64(0.0005349192640053957) - rollout_corr/k3_kl:np.float64(0.0004940067541099324) - rollout_corr/rollout_ppl:np.float64(1.2394655891694129) - rollout_corr/rollout_log_ppl:np.float64(0.19906326734781032) - rollout_corr/log_ppl_diff:np.float64(0.0005349202401703224) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011775598395615816) - rollout_corr/log_ppl_diff_max:np.float64(0.005937367677688599) - rollout_corr/log_ppl_diff_min:np.float64(-0.004511777311563492) - rollout_corr/ppl_ratio:np.float64(1.000536180799827) - rollout_corr/chi2_token:np.float64(0.0009251013398170471) - rollout_corr/chi2_seq:np.float64(1.390581222018227) - actor/pg_loss:np.float64(4.893657751381397e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018130719581677113) - actor/ppo_kl:np.float64(-2.2069814953340483e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15835927985608578) - perf/mfu/actor:np.float64(0.08403507336330182) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.51084518432617) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6375) - val-aux/sciknoweval/reward/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10267672443906181) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8312625) - val-aux/sciknoweval/reward/best@16/std:np.float64
(TaskRunner pid=2049544) Training Progress: 60%|██████ | 60/100 [46:34<50:26, 75.65s/it]
(TaskRunner pid=2049544) step:61 - global_seqlen/min:22238 - global_seqlen/max:39486 - global_seqlen/minmax_diff:17248 - global_seqlen/balanced_min:29924 - global_seqlen/balanced_max:30089 - global_seqlen/mean:29970.875 - actor/entropy:0.23904968798160553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5477294921875 - training/rollout_probs_diff_mean:0.003164143767207861 - training/rollout_probs_diff_std:0.00985527690500021 - training/rollout_actor_probs_pearson_corr:0.9991676211357117 - rollout_corr/rollout_is_mean:1.0000258684158325 - rollout_corr/rollout_is_ratio_fraction_high:1.1471639481897e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.720745967759285e-05 - rollout_corr/rollout_is_max:4.02189826965332 - rollout_corr/rollout_is_min:0.26447275280952454 - rollout_corr/rollout_is_std:0.030506834387779236 - rollout_corr/rollout_is_eff_sample_size:0.9990700794243639 - rollout_corr/rollout_is_seq_mean:1.0000176429748535 - rollout_corr/rollout_is_seq_std:0.0016575142508372664 - rollout_corr/rollout_is_seq_max:1.0063189268112183 - rollout_corr/rollout_is_seq_min:0.9920255541801453 - rollout_corr/rollout_is_seq_max_deviation:0.007974445819854736 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.25040641753003) - rollout_corr/training_log_ppl:np.float64(0.20799311240261886) - rollout_corr/kl:np.float64(0.0005623800304297433) - rollout_corr/k3_kl:np.float64(0.0005192289710862497) - rollout_corr/rollout_ppl:np.float64(1.2496482259593904) - rollout_corr/rollout_log_ppl:np.float64(0.20743073029007064) - rollout_corr/log_ppl_diff:np.float64(0.0005623821125482209) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012774796996382065) - rollout_corr/log_ppl_diff_max:np.float64(0.009329631924629211) - rollout_corr/log_ppl_diff_min:np.float64(-0.007050305604934692) - rollout_corr/ppl_ratio:np.float64(1.0005640257149935) - rollout_corr/chi2_token:np.float64(0.0009777885861694813) - rollout_corr/chi2_seq:np.float64(0.3470004736445844) - actor/pg_loss:np.float64(-0.0002099055564031005) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002631939267985217) - actor/ppo_kl:np.float64(6.142829495781044e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23834596201777458) - perf/mfu/actor:np.float64(0.08600884737044563) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.00771045684814) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007454414386302233 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.007454414386302233 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:681.02734375 - response_length/max:3128.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:681.02734375 - response_length_non_aborted/max:3128.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015644170343875885 - timing_s/agent_loop/generate_sequences/min:np.float64(0.93056246265769) - timing_s/agent_loop/generate_sequences/max:np.float64(20.83841021731496) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.5293213194963755) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.83841021731496) - timing_s/agent_
(TaskRunner pid=2049544) Training Progress: 61%|██████ | 61/100 [47:14<42:10, 64.88s/it]
(TaskRunner pid=2049544) step:62 - global_seqlen/min:24330 - global_seqlen/max:43162 - global_seqlen/minmax_diff:18832 - global_seqlen/balanced_min:34742 - global_seqlen/balanced_max:34794 - global_seqlen/mean:34771.375 - actor/entropy:0.2732798457145691 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4058573246002197 - training/rollout_probs_diff_mean:0.00327512389048934 - training/rollout_probs_diff_std:0.009542555548250675 - training/rollout_actor_probs_pearson_corr:0.9991864562034607 - rollout_corr/rollout_is_mean:0.9999688267707825 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.467516540287761e-05 - rollout_corr/rollout_is_max:1.9362671375274658 - rollout_corr/rollout_is_min:0.37468504905700684 - rollout_corr/rollout_is_std:0.030330481007695198 - rollout_corr/rollout_is_eff_sample_size:0.9990807884305588 - rollout_corr/rollout_is_seq_mean:0.9999213814735413 - rollout_corr/rollout_is_seq_std:0.0014323026407510042 - rollout_corr/rollout_is_seq_max:1.0065138339996338 - rollout_corr/rollout_is_seq_min:0.9906108975410461 - rollout_corr/rollout_is_seq_max_deviation:0.009389102458953857 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2884436114691198) - rollout_corr/training_log_ppl:np.float64(0.23855835630820366) - rollout_corr/kl:np.float64(0.0005924100263499099) - rollout_corr/k3_kl:np.float64(0.0005057925414178044) - rollout_corr/rollout_ppl:np.float64(1.2876463397406042) - rollout_corr/rollout_log_ppl:np.float64(0.23796594546729466) - rollout_corr/log_ppl_diff:np.float64(0.0005924108409089968) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011562938307179138) - rollout_corr/log_ppl_diff_max:np.float64(0.009115807712078094) - rollout_corr/log_ppl_diff_min:np.float64(-0.0043695345520973206) - rollout_corr/ppl_ratio:np.float64(1.0005937188398093) - rollout_corr/chi2_token:np.float64(0.0008554272353649139) - rollout_corr/chi2_seq:np.float64(0.8644126462750137) - actor/pg_loss:np.float64(-5.7160970754921436e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029845362780633877) - actor/ppo_kl:np.float64(3.140159503445261e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22896912693977356) - perf/mfu/actor:np.float64(0.09839975519482527) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0562973022461) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022515371441841125 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022515371441841125 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:798.54296875 - response_length/max:3135.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:798.54296875 - response_length_non_aborted/max:3135.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014179199934005737 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8601270448416471) - timing_s/agent_loop/generate_sequences/max:np.float64(20.932036239653826) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.75760180906218) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.932036239653826) - timing_s/agent_loop
(TaskRunner pid=2049544) Training Progress: 62%|██████▏ | 62/100 [47:55<36:26, 57.55s/it]
(TaskRunner pid=2049544) step:63 - global_seqlen/min:21359 - global_seqlen/max:35870 - global_seqlen/minmax_diff:14511 - global_seqlen/balanced_min:26722 - global_seqlen/balanced_max:27254 - global_seqlen/mean:26812.0 - actor/entropy:0.27676644921302795 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9831006526947021 - training/rollout_probs_diff_mean:0.0035199327394366264 - training/rollout_probs_diff_std:0.010530827566981316 - training/rollout_actor_probs_pearson_corr:0.9990900158882141 - rollout_corr/rollout_is_mean:0.9999789595603943 - rollout_corr/rollout_is_ratio_fraction_high:1.3679142284672707e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.419785571168177e-05 - rollout_corr/rollout_is_max:2.8213930130004883 - rollout_corr/rollout_is_min:0.011052249930799007 - rollout_corr/rollout_is_std:0.031423088163137436 - rollout_corr/rollout_is_eff_sample_size:0.9990135634724805 - rollout_corr/rollout_is_seq_mean:1.0000073909759521 - rollout_corr/rollout_is_seq_std:0.001668434706516564 - rollout_corr/rollout_is_seq_max:1.0121815204620361 - rollout_corr/rollout_is_seq_min:0.9929383993148804 - rollout_corr/rollout_is_seq_max_deviation:0.012181520462036133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2705106027424335) - rollout_corr/training_log_ppl:np.float64(0.22211689058167394) - rollout_corr/kl:np.float64(0.0004739185238458177) - rollout_corr/k3_kl:np.float64(0.0005098591802266128) - rollout_corr/rollout_ppl:np.float64(1.2698905346915126) - rollout_corr/rollout_log_ppl:np.float64(0.2216429725958733) - rollout_corr/log_ppl_diff:np.float64(0.0004739179858006537) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012847277976106852) - rollout_corr/log_ppl_diff_max:np.float64(0.0076948776841163635) - rollout_corr/log_ppl_diff_min:np.float64(-0.010769776999950409) - rollout_corr/ppl_ratio:np.float64(1.0004755130503327) - rollout_corr/chi2_token:np.float64(0.001110320445150137) - rollout_corr/chi2_seq:np.float64(0.7298937705345452) - actor/pg_loss:np.float64(-6.4569758251309395e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017891856134610862) - actor/ppo_kl:np.float64(-4.981012998683809e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19124767929315567) - perf/mfu/actor:np.float64(0.07698739538780973) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.1104507446289) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018338599475100636 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0018338599475100636 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:571.125 - response_length/max:3010.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.125 - response_length_non_aborted/max:3010.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001445487141609192 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8224132973700762) - timing_s/agent_loop/generate_sequences/max:np.float64(18.348940243944526) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.530061706544075) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.348940243944526) - timing_s/age
(TaskRunner pid=2049544) Training Progress: 63%|██████▎ | 63/100 [48:33<31:54, 51.74s/it]
(TaskRunner pid=2049544) step:64 - global_seqlen/min:26498 - global_seqlen/max:40322 - global_seqlen/minmax_diff:13824 - global_seqlen/balanced_min:31563 - global_seqlen/balanced_max:31618 - global_seqlen/mean:31602.875 - actor/entropy:0.28700023889541626 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535178303718567 - training/rollout_probs_diff_mean:0.0033518956042826176 - training/rollout_probs_diff_std:0.009729590266942978 - training/rollout_actor_probs_pearson_corr:0.9992855787277222 - rollout_corr/rollout_is_mean:1.0000327825546265 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0833825399458874e-05 - rollout_corr/rollout_is_max:1.9838535785675049 - rollout_corr/rollout_is_min:0.4158313572406769 - rollout_corr/rollout_is_std:0.03056929260492325 - rollout_corr/rollout_is_eff_sample_size:0.9990665098066491 - rollout_corr/rollout_is_seq_mean:0.9999979734420776 - rollout_corr/rollout_is_seq_std:0.0013619516976177692 - rollout_corr/rollout_is_seq_max:1.0039030313491821 - rollout_corr/rollout_is_seq_min:0.9954256415367126 - rollout_corr/rollout_is_seq_max_deviation:0.0045743584632873535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2764493175782263) - rollout_corr/training_log_ppl:np.float64(0.22693806317693088) - rollout_corr/kl:np.float64(0.00046152420509582015) - rollout_corr/k3_kl:np.float64(0.0005074531937054871) - rollout_corr/rollout_ppl:np.float64(1.275848121382296) - rollout_corr/rollout_log_ppl:np.float64(0.22647653929016087) - rollout_corr/log_ppl_diff:np.float64(0.00046152388677001) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011273344280198216) - rollout_corr/log_ppl_diff_max:np.float64(0.00530514121055603) - rollout_corr/log_ppl_diff_min:np.float64(-0.005483843386173248) - rollout_corr/ppl_ratio:np.float64(1.0004626805894077) - rollout_corr/chi2_token:np.float64(0.0011098254472017288) - rollout_corr/chi2_seq:np.float64(1.8452809148002416) - actor/pg_loss:np.float64(-0.00011544267181307077) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002127377759961746) - actor/ppo_kl:np.float64(-4.476623224114462e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18739718943834305) - perf/mfu/actor:np.float64(0.08994342294998352) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.98906707763672) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013745415955781937 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013745415955781937 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:721.12109375 - response_length/max:2725.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:721.12109375 - response_length_non_aborted/max:2725.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.722448885440826e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.080433465540409) - timing_s/agent_loop/generate_sequences/max:np.float64(17.78822729177773) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.979315571334155) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.78822729177773) - timing_s/agent_lo
(TaskRunner pid=2049544) Training Progress: 64%|██████▍ | 64/100 [49:10<28:23, 47.32s/it]
(TaskRunner pid=2049544) step:65 - global_seqlen/min:13817 - global_seqlen/max:44402 - global_seqlen/minmax_diff:30585 - global_seqlen/balanced_min:27087 - global_seqlen/balanced_max:27398 - global_seqlen/mean:27214.25 - actor/entropy:0.2703191936016083 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730000019073486 - training/rollout_probs_diff_mean:0.003337751142680645 - training/rollout_probs_diff_std:0.009712927974760532 - training/rollout_actor_probs_pearson_corr:0.999212384223938 - rollout_corr/rollout_is_mean:0.9999141097068787 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3306897017173469e-05 - rollout_corr/rollout_is_max:1.8171900510787964 - rollout_corr/rollout_is_min:0.37625378370285034 - rollout_corr/rollout_is_std:0.030340304598212242 - rollout_corr/rollout_is_eff_sample_size:0.999079955499621 - rollout_corr/rollout_is_seq_mean:1.0000232458114624 - rollout_corr/rollout_is_seq_std:0.001954729435965419 - rollout_corr/rollout_is_seq_max:1.0170458555221558 - rollout_corr/rollout_is_seq_min:0.9947062730789185 - rollout_corr/rollout_is_seq_max_deviation:0.01704585552215576 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2311511896550655) - rollout_corr/training_log_ppl:np.float64(0.1959650574644911) - rollout_corr/kl:np.float64(0.0005068062779431948) - rollout_corr/k3_kl:np.float64(0.0005566824616494159) - rollout_corr/rollout_ppl:np.float64(1.2305298056453466) - rollout_corr/rollout_log_ppl:np.float64(0.1954582512989873) - rollout_corr/log_ppl_diff:np.float64(0.0005068061655038036) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013243403445812874) - rollout_corr/log_ppl_diff_max:np.float64(0.007357165217399597) - rollout_corr/log_ppl_diff_min:np.float64(-0.020653128623962402) - rollout_corr/ppl_ratio:np.float64(1.0005090332124382) - rollout_corr/chi2_token:np.float64(0.0013379347510635853) - rollout_corr/chi2_seq:np.float64(0.5852078427560627) - actor/pg_loss:np.float64(5.363975651562214e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001393496023638363) - actor/ppo_kl:np.float64(4.77582690940892e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17264313623309135) - perf/mfu/actor:np.float64(0.07762399981838528) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82825469970703) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034032389521598816 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034032389521598816 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:587.1015625 - response_length/max:3738.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:587.1015625 - response_length_non_aborted/max:3738.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.999820470809937e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0192465409636497) - timing_s/agent_loop/generate_sequences/max:np.float64(20.993049459531903) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.240891806744912) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.993049459531903) - timing_s/agent_loop/slowest/to
(TaskRunner pid=2049544) Training Progress: 65%|██████▌ | 65/100 [49:50<26:17, 45.07s/it]
(TaskRunner pid=2049544) step:66 - global_seqlen/min:26398 - global_seqlen/max:47072 - global_seqlen/minmax_diff:20674 - global_seqlen/balanced_min:34031 - global_seqlen/balanced_max:34100 - global_seqlen/mean:34083.75 - actor/entropy:0.32043859362602234 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4526241421699524 - training/rollout_probs_diff_mean:0.003443863708525896 - training/rollout_probs_diff_std:0.009328007698059082 - training/rollout_actor_probs_pearson_corr:0.999345064163208 - rollout_corr/rollout_is_mean:0.999977171421051 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9896735466318205e-05 - rollout_corr/rollout_is_max:1.9577182531356812 - rollout_corr/rollout_is_min:0.13161711394786835 - rollout_corr/rollout_is_std:0.030255712568759918 - rollout_corr/rollout_is_eff_sample_size:0.9990853100798829 - rollout_corr/rollout_is_seq_mean:1.0000357627868652 - rollout_corr/rollout_is_seq_std:0.0014041649410501122 - rollout_corr/rollout_is_seq_max:1.0082844495773315 - rollout_corr/rollout_is_seq_min:0.9953852295875549 - rollout_corr/rollout_is_seq_max_deviation:0.008284449577331543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3104135780595243) - rollout_corr/training_log_ppl:np.float64(0.25584780907956883) - rollout_corr/kl:np.float64(0.0005620165548441136) - rollout_corr/k3_kl:np.float64(0.0004995921321437891) - rollout_corr/rollout_ppl:np.float64(1.309668519999832) - rollout_corr/rollout_log_ppl:np.float64(0.2552857908885926) - rollout_corr/log_ppl_diff:np.float64(0.0005620181909762323) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011583589657675475) - rollout_corr/log_ppl_diff_max:np.float64(0.005684643983840942) - rollout_corr/log_ppl_diff_min:np.float64(-0.006929904222488403) - rollout_corr/ppl_ratio:np.float64(1.000563226873055) - rollout_corr/chi2_token:np.float64(0.0008636738639324903) - rollout_corr/chi2_seq:np.float64(0.8563104090280831) - actor/pg_loss:np.float64(0.0001647728495299816) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017283655381561402) - actor/ppo_kl:np.float64(0.00013175434309165013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1924263872206211) - perf/mfu/actor:np.float64(0.09599912196819169) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.91796875) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000762293697334826 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000762293697334826 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:785.3046875 - response_length/max:3216.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:785.3046875 - response_length_non_aborted/max:3216.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.582083344459534e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2348940391093493) - timing_s/agent_loop/generate_sequences/max:np.float64(21.131182158365846) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.555415921531676) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.131182158365846) - timing_s/agent_loop/slowest/to
(TaskRunner pid=2049544) Training Progress: 66%|██████▌ | 66/100 [50:30<24:42, 43.61s/it]
(TaskRunner pid=2049544) step:67 - global_seqlen/min:14820 - global_seqlen/max:39555 - global_seqlen/minmax_diff:24735 - global_seqlen/balanced_min:25556 - global_seqlen/balanced_max:26212 - global_seqlen/mean:25672.375 - actor/entropy:0.27359628677368164 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46210977435112 - training/rollout_probs_diff_mean:0.0033997194841504097 - training/rollout_probs_diff_std:0.009928795509040356 - training/rollout_actor_probs_pearson_corr:0.9991419315338135 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1833584469277412e-05 - rollout_corr/rollout_is_max:1.9785306453704834 - rollout_corr/rollout_is_min:0.3678695261478424 - rollout_corr/rollout_is_std:0.031190816313028336 - rollout_corr/rollout_is_eff_sample_size:0.9990281975159966 - rollout_corr/rollout_is_seq_mean:1.0001834630966187 - rollout_corr/rollout_is_seq_std:0.002221266273409128 - rollout_corr/rollout_is_seq_max:1.0159006118774414 - rollout_corr/rollout_is_seq_min:0.9894429445266724 - rollout_corr/rollout_is_seq_max_deviation:0.015900611877441406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3023830708116293) - rollout_corr/training_log_ppl:np.float64(0.2422099069954129) - rollout_corr/kl:np.float64(0.00026683149832429365) - rollout_corr/k3_kl:np.float64(0.000541509634160775) - rollout_corr/rollout_ppl:np.float64(1.301996021065861) - rollout_corr/rollout_log_ppl:np.float64(0.241943074419396) - rollout_corr/log_ppl_diff:np.float64(0.0002668325760168955) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014319048641482368) - rollout_corr/log_ppl_diff_max:np.float64(0.009274959564208984) - rollout_corr/log_ppl_diff_min:np.float64(-0.0138968825340271) - rollout_corr/ppl_ratio:np.float64(1.0002689892426133) - rollout_corr/chi2_token:np.float64(0.0016502379439771175) - rollout_corr/chi2_seq:np.float64(1.019609775627032) - actor/pg_loss:np.float64(-5.0529371947050095e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016554813828406623) - actor/ppo_kl:np.float64(-7.827897409384832e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17443475499749184) - perf/mfu/actor:np.float64(0.07407053273139065) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82710266113281) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01963839866220951 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01963839866220951 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:536.73046875 - response_length/max:3340.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:536.73046875 - response_length_non_aborted/max:3340.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00023074820637702942 - timing_s/agent_loop/generate_sequences/min:np.float64(1.033557141199708) - timing_s/agent_loop/generate_sequences/max:np.float64(19.718276670202613) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.309584286827885) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.718276670202613) - timing_s/agent_loop/slowest/
(TaskRunner pid=2049544) Training Progress: 67%|██████▋ | 67/100 [51:09<23:14, 42.26s/it]
(TaskRunner pid=2049544) step:68 - global_seqlen/min:21420 - global_seqlen/max:36370 - global_seqlen/minmax_diff:14950 - global_seqlen/balanced_min:26998 - global_seqlen/balanced_max:27019 - global_seqlen/mean:27007.25 - actor/entropy:0.2508779466152191 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4620983600616455 - training/rollout_probs_diff_mean:0.003281368175521493 - training/rollout_probs_diff_std:0.009971853345632553 - training/rollout_actor_probs_pearson_corr:0.9991183876991272 - rollout_corr/rollout_is_mean:0.9999003410339355 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.37541350745596e-05 - rollout_corr/rollout_is_max:1.727583885192871 - rollout_corr/rollout_is_min:0.3678779602050781 - rollout_corr/rollout_is_std:0.03009769693017006 - rollout_corr/rollout_is_eff_sample_size:0.9990948294754013 - rollout_corr/rollout_is_seq_mean:0.999991238117218 - rollout_corr/rollout_is_seq_std:0.0015605590306222439 - rollout_corr/rollout_is_seq_max:1.005854606628418 - rollout_corr/rollout_is_seq_min:0.9957244396209717 - rollout_corr/rollout_is_seq_max_deviation:0.005854606628417969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2427316303364933) - rollout_corr/training_log_ppl:np.float64(0.2052579416122171) - rollout_corr/kl:np.float64(0.0006245356830429927) - rollout_corr/k3_kl:np.float64(0.0004809370777678623) - rollout_corr/rollout_ppl:np.float64(1.2419258821755648) - rollout_corr/rollout_log_ppl:np.float64(0.20463340327114565) - rollout_corr/log_ppl_diff:np.float64(0.0006245383410714567) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013257119280751795) - rollout_corr/log_ppl_diff_max:np.float64(0.005434125661849976) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038042664527893066) - rollout_corr/ppl_ratio:np.float64(1.0006259840447456) - rollout_corr/chi2_token:np.float64(0.0006594203878194094) - rollout_corr/chi2_seq:np.float64(0.44312575249932706) - actor/pg_loss:np.float64(-5.7110912166535854e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002576871158908034) - actor/ppo_kl:np.float64(0.000146509498037517) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1650708168745041) - perf/mfu/actor:np.float64(0.07864794728282518) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82149505615234) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00015104975318536162 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00015104975318536162 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:578.6328125 - response_length/max:1942.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:578.6328125 - response_length_non_aborted/max:1942.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.873881936073303e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0787797346711159) - timing_s/agent_loop/generate_sequences/max:np.float64(13.899392530322075) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.687431053593173) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.899392530322075) - timing_s/agent_loop/slow
(TaskRunner pid=2049544) Training Progress: 68%|██████▊ | 68/100 [51:41<20:57, 39.29s/it]
(TaskRunner pid=2049544) step:69 - global_seqlen/min:23146 - global_seqlen/max:36753 - global_seqlen/minmax_diff:13607 - global_seqlen/balanced_min:29343 - global_seqlen/balanced_max:29374 - global_seqlen/mean:29363.125 - actor/entropy:0.2744224965572357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491748213768005 - training/rollout_probs_diff_mean:0.0033163181506097317 - training/rollout_probs_diff_std:0.00957618560642004 - training/rollout_actor_probs_pearson_corr:0.9992418885231018 - rollout_corr/rollout_is_mean:0.9999679327011108 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9769123792648315 - rollout_corr/rollout_is_min:0.547244668006897 - rollout_corr/rollout_is_std:0.030115514993667603 - rollout_corr/rollout_is_eff_sample_size:0.9990938775276863 - rollout_corr/rollout_is_seq_mean:1.000019907951355 - rollout_corr/rollout_is_seq_std:0.001533227856270969 - rollout_corr/rollout_is_seq_max:1.0057690143585205 - rollout_corr/rollout_is_seq_min:0.990231990814209 - rollout_corr/rollout_is_seq_max_deviation:0.009768009185791016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2841168828308582) - rollout_corr/training_log_ppl:np.float64(0.23235505366756115) - rollout_corr/kl:np.float64(0.0005125461427646361) - rollout_corr/k3_kl:np.float64(0.0005065118215270559) - rollout_corr/rollout_ppl:np.float64(1.2834058650769293) - rollout_corr/rollout_log_ppl:np.float64(0.2318425054399995) - rollout_corr/log_ppl_diff:np.float64(0.0005125482275616378) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012335704232100397) - rollout_corr/log_ppl_diff_max:np.float64(0.011537343263626099) - rollout_corr/log_ppl_diff_min:np.float64(-0.005053341388702393) - rollout_corr/ppl_ratio:np.float64(1.0005140437278897) - rollout_corr/chi2_token:np.float64(0.00102224200963974) - rollout_corr/chi2_seq:np.float64(0.8634938539471477) - actor/pg_loss:np.float64(-5.3855590522289276e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000181908647164164) - actor/ppo_kl:np.float64(5.1798504737021744e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18398911505937576) - perf/mfu/actor:np.float64(0.08486034559374442) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82257843017578) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015627924352884293 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015627924352884293 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:646.87890625 - response_length/max:2353.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.87890625 - response_length_non_aborted/max:2353.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.917755722999573e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.203237522393465) - timing_s/agent_loop/generate_sequences/max:np.float64(16.094254648312926) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.346816844648856) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.094254648312926) - timing_s/agent_loop/slowest/tool_calls:np.fl
(TaskRunner pid=2049544) Training Progress: 69%|██████▉ | 69/100 [52:16<19:37, 37.99s/it]
(TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70}
(TaskRunner pid=2049544) validation generation end
(TaskRunner pid=2049544) [prompt] system
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2049544) <reasoning>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) A
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) user
(TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) A: 50 pC
(TaskRunner pid=2049544) B: 56 pC
(TaskRunner pid=2049544) C: 60 pC
(TaskRunner pid=2049544) D: 64 pC
(TaskRunner pid=2049544) Please reason step by step.
(TaskRunner pid=2049544) assistant
(TaskRunner pid=2049544) <think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) [response] <reasoning>
(TaskRunner pid=2049544) We are given:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $
(TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $
(TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) We are to find the charge $ q $ using the formula:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) E = k \frac{q}{r^2}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Rearranging for $ q $:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Substitute the values:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2049544) = \frac{2.0 \cdot 0.25}{8.99 \times 10^9}
(TaskRunner pid=2049544) = \frac{0.5}{8.99 \times 10^9}
(TaskRunner pid=2049544) \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C):
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) The closest answer is **B: 56 pC**.
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) B
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544) [ground_truth] B
(TaskRunner pid=2049544) [score] 1.0
(TaskRunner pid=2049544) [acc] 1.0
(TaskRunner pid=2049544) [pred] B
(TaskRunner pid=2049544) [incorrect_format] 1
(TaskRunner pid=2049544) [feedback]
(TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70
(WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor
(TaskRunner pid=2049544) step:70 - global_seqlen/min:18463 - global_seqlen/max:39933 - global_seqlen/minmax_diff:21470 - global_seqlen/balanced_min:27119 - global_seqlen/balanced_max:27177 - global_seqlen/mean:27165.625 - actor/entropy:0.34878426790237427 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35282185673713684 - training/rollout_probs_diff_mean:0.003718312829732895 - training/rollout_probs_diff_std:0.009631498716771603 - training/rollout_actor_probs_pearson_corr:0.999345600605011 - rollout_corr/rollout_is_mean:1.000012993812561 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0160612621111795e-05 - rollout_corr/rollout_is_max:1.975199818611145 - rollout_corr/rollout_is_min:0.19730398058891296 - rollout_corr/rollout_is_std:0.03194605931639671 - rollout_corr/rollout_is_eff_sample_size:0.9989806087004571 - rollout_corr/rollout_is_seq_mean:1.0000489950180054 - rollout_corr/rollout_is_seq_std:0.0017589139752089977 - rollout_corr/rollout_is_seq_max:1.0087381601333618 - rollout_corr/rollout_is_seq_min:0.992647647857666 - rollout_corr/rollout_is_seq_max_deviation:0.008738160133361816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3483932986855507) - rollout_corr/training_log_ppl:np.float64(0.2762820063435356) - rollout_corr/kl:np.float64(0.0006076907076355553) - rollout_corr/k3_kl:np.float64(0.0005228528713416836) - rollout_corr/rollout_ppl:np.float64(1.347519775852561) - rollout_corr/rollout_log_ppl:np.float64(0.27567431312490953) - rollout_corr/log_ppl_diff:np.float64(0.0006076932186260819) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012910107034258544) - rollout_corr/log_ppl_diff_max:np.float64(0.009554415941238403) - rollout_corr/log_ppl_diff_min:np.float64(-0.007596790790557861) - rollout_corr/ppl_ratio:np.float64(1.0006093357224017) - rollout_corr/chi2_token:np.float64(0.0008673274423927069) - rollout_corr/chi2_seq:np.float64(0.4545943159610033) - actor/pg_loss:np.float64(4.701910074800253e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021670004468887782) - actor/ppo_kl:np.float64(0.0001372517156355002) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2546451985836029) - perf/mfu/actor:np.float64(0.07849234372480743) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.89488983154297) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09773915943586196) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8506) - val-aux/sciknoweval/reward/best@16/std:np.f
(TaskRunner pid=2049544) Training Progress: 70%|███████ | 70/100 [54:53<36:48, 73.62s/it]
(TaskRunner pid=2049544) step:71 - global_seqlen/min:20204 - global_seqlen/max:38376 - global_seqlen/minmax_diff:18172 - global_seqlen/balanced_min:28759 - global_seqlen/balanced_max:29243 - global_seqlen/mean:28856.5 - actor/entropy:0.32371971011161804 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35350361466407776 - training/rollout_probs_diff_mean:0.003485669381916523 - training/rollout_probs_diff_std:0.009408628568053246 - training/rollout_actor_probs_pearson_corr:0.9993824362754822 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:6.257665518205613e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.257665518205613e-06 - rollout_corr/rollout_is_max:3.121652126312256 - rollout_corr/rollout_is_min:0.4736731946468353 - rollout_corr/rollout_is_std:0.03072100132703781 - rollout_corr/rollout_is_eff_sample_size:0.9990571099353601 - rollout_corr/rollout_is_seq_mean:1.0000032186508179 - rollout_corr/rollout_is_seq_std:0.0015029723290354013 - rollout_corr/rollout_is_seq_max:1.0061728954315186 - rollout_corr/rollout_is_seq_min:0.9937267303466797 - rollout_corr/rollout_is_seq_max_deviation:0.0062732696533203125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3052765680477023) - rollout_corr/training_log_ppl:np.float64(0.24597317739971913) - rollout_corr/kl:np.float64(0.00043000688371730433) - rollout_corr/k3_kl:np.float64(0.000540082649621354) - rollout_corr/rollout_ppl:np.float64(1.3047043732367456) - rollout_corr/rollout_log_ppl:np.float64(0.2455431708949618) - rollout_corr/log_ppl_diff:np.float64(0.0004300065047573298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012972943659406155) - rollout_corr/log_ppl_diff_max:np.float64(0.00797504186630249) - rollout_corr/log_ppl_diff_min:np.float64(-0.0062088742852211) - rollout_corr/ppl_ratio:np.float64(1.0004315802361816) - rollout_corr/chi2_token:np.float64(0.0013474705629050732) - rollout_corr/chi2_seq:np.float64(1.4930782679002732) - actor/pg_loss:np.float64(0.0001349709928035736) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038389957057916035) - actor/ppo_kl:np.float64(-3.772802561208266e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17576251924037933) - perf/mfu/actor:np.float64(0.08244675822096596) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.2421646118164) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009262127801775932 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009262127801775932 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:624.234375 - response_length/max:3404.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:624.234375 - response_length_non_aborted/max:3404.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012946873903274536 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2765609622001648) - timing_s/agent_loop/generate_sequences/max:np.float64(20.10214041545987) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.958212114004709) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.10214041545987) - timing_s/agent_loop/
(TaskRunner pid=2049544) Training Progress: 71%|███████ | 71/100 [55:32<30:32, 63.19s/it]
(TaskRunner pid=2049544) step:72 - global_seqlen/min:24019 - global_seqlen/max:35372 - global_seqlen/minmax_diff:11353 - global_seqlen/balanced_min:30681 - global_seqlen/balanced_max:31067 - global_seqlen/mean:30756.75 - actor/entropy:0.2652339041233063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5334677696228027 - training/rollout_probs_diff_mean:0.003048401791602373 - training/rollout_probs_diff_std:0.009212449193000793 - training/rollout_actor_probs_pearson_corr:0.9992727637290955 - rollout_corr/rollout_is_mean:1.0000463724136353 - rollout_corr/rollout_is_ratio_fraction_high:1.1159842870256398e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.6739762941142544e-05 - rollout_corr/rollout_is_max:2.703545570373535 - rollout_corr/rollout_is_min:0.2786954641342163 - rollout_corr/rollout_is_std:0.02862177975475788 - rollout_corr/rollout_is_eff_sample_size:0.9991814643117487 - rollout_corr/rollout_is_seq_mean:1.0000512599945068 - rollout_corr/rollout_is_seq_std:0.0012803305871784687 - rollout_corr/rollout_is_seq_max:1.005553960800171 - rollout_corr/rollout_is_seq_min:0.9965124726295471 - rollout_corr/rollout_is_seq_max_deviation:0.0055539608001708984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.272761202417314) - rollout_corr/training_log_ppl:np.float64(0.2287760855979286) - rollout_corr/kl:np.float64(0.0003352667156821809) - rollout_corr/k3_kl:np.float64(0.0004365081685619998) - rollout_corr/rollout_ppl:np.float64(1.2723114336840808) - rollout_corr/rollout_log_ppl:np.float64(0.22844081881339662) - rollout_corr/log_ppl_diff:np.float64(0.00033526678453199565) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010565653501544148) - rollout_corr/log_ppl_diff_max:np.float64(0.00969955325126648) - rollout_corr/log_ppl_diff_min:np.float64(-0.005003869533538818) - rollout_corr/ppl_ratio:np.float64(1.0003363261930645) - rollout_corr/chi2_token:np.float64(0.0010922125075012445) - rollout_corr/chi2_seq:np.float64(1.312244726344943) - actor/pg_loss:np.float64(8.467817679047585e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000129777238043971) - actor/ppo_kl:np.float64(-3.5065210759910315e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1640554666519165) - perf/mfu/actor:np.float64(0.08815191325739583) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.06222915649414) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006141400896012783 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006141400896012783 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:700.0546875 - response_length/max:3313.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:700.0546875 - response_length_non_aborted/max:3313.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013216771185398102 - timing_s/agent_loop/generate_sequences/min:np.float64(2.07568240724504) - timing_s/agent_loop/generate_sequences/max:np.float64(20.67868852801621) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.946097736770753) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.67868852801621) - timing_s/a
(TaskRunner pid=2049544) Training Progress: 72%|███████▏ | 72/100 [56:11<26:10, 56.08s/it]
(TaskRunner pid=2049544) step:73 - global_seqlen/min:21925 - global_seqlen/max:46347 - global_seqlen/minmax_diff:24422 - global_seqlen/balanced_min:36563 - global_seqlen/balanced_max:36628 - global_seqlen/mean:36610.375 - actor/entropy:0.3390262722969055 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43396425247192383 - training/rollout_probs_diff_mean:0.0032661668956279755 - training/rollout_probs_diff_std:0.008885001763701439 - training/rollout_actor_probs_pearson_corr:0.9994308948516846 - rollout_corr/rollout_is_mean:0.9999864101409912 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7025687813875265e-05 - rollout_corr/rollout_is_max:1.9699597358703613 - rollout_corr/rollout_is_min:0.2820616662502289 - rollout_corr/rollout_is_std:0.029777150601148605 - rollout_corr/rollout_is_eff_sample_size:0.9991141068069422 - rollout_corr/rollout_is_seq_mean:0.9999426007270813 - rollout_corr/rollout_is_seq_std:0.0015595207223668694 - rollout_corr/rollout_is_seq_max:1.0064419507980347 - rollout_corr/rollout_is_seq_min:0.992620587348938 - rollout_corr/rollout_is_seq_max_deviation:0.007379412651062012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4027120932005346) - rollout_corr/training_log_ppl:np.float64(0.31371801071509253) - rollout_corr/kl:np.float64(0.000663529713321509) - rollout_corr/k3_kl:np.float64(0.0005334890627182176) - rollout_corr/rollout_ppl:np.float64(1.4016525098122656) - rollout_corr/rollout_log_ppl:np.float64(0.3130544779851334) - rollout_corr/log_ppl_diff:np.float64(0.0006635327299591154) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013385674392338842) - rollout_corr/log_ppl_diff_max:np.float64(0.008049964904785156) - rollout_corr/log_ppl_diff_min:np.float64(-0.005723848938941956) - rollout_corr/ppl_ratio:np.float64(1.0006653110031039) - rollout_corr/chi2_token:np.float64(0.0007862187922000885) - rollout_corr/chi2_seq:np.float64(0.7408852751832455) - actor/pg_loss:np.float64(-9.118067100644112e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025507958866910485) - actor/ppo_kl:np.float64(9.864521248204028e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22031916305422783) - perf/mfu/actor:np.float64(0.10239718785831436) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.18194198608398) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006957425735890865 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006957425735890865 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:867.23046875 - response_length/max:3153.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:867.23046875 - response_length_non_aborted/max:3153.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010944902896881104 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9543074388056993) - timing_s/agent_loop/generate_sequences/max:np.float64(21.861840976402164) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.284751043880533) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.861840976402164) - timing_s/agent_l
(TaskRunner pid=2049544) Training Progress: 73%|███████▎ | 73/100 [56:52<23:12, 51.57s/it]
(TaskRunner pid=2049544) step:74 - global_seqlen/min:18401 - global_seqlen/max:45796 - global_seqlen/minmax_diff:27395 - global_seqlen/balanced_min:30702 - global_seqlen/balanced_max:30981 - global_seqlen/mean:30746.0 - actor/entropy:0.31715160608291626 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4399150013923645 - training/rollout_probs_diff_mean:0.0034521878696978092 - training/rollout_probs_diff_std:0.009627916850149632 - training/rollout_actor_probs_pearson_corr:0.9993271231651306 - rollout_corr/rollout_is_mean:1.0001097917556763 - rollout_corr/rollout_is_ratio_fraction_high:1.1016612916137092e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.652492028370034e-05 - rollout_corr/rollout_is_max:2.2787067890167236 - rollout_corr/rollout_is_min:0.41993895173072815 - rollout_corr/rollout_is_std:0.030713239684700966 - rollout_corr/rollout_is_eff_sample_size:0.9990577048586795 - rollout_corr/rollout_is_seq_mean:1.000206708908081 - rollout_corr/rollout_is_seq_std:0.002247885102406144 - rollout_corr/rollout_is_seq_max:1.0164759159088135 - rollout_corr/rollout_is_seq_min:0.9938144683837891 - rollout_corr/rollout_is_seq_max_deviation:0.016475915908813477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3208666159771383) - rollout_corr/training_log_ppl:np.float64(0.2606007744325325) - rollout_corr/kl:np.float64(0.0003702222969650393) - rollout_corr/k3_kl:np.float64(0.000720645587279023) - rollout_corr/rollout_ppl:np.float64(1.3202885040082037) - rollout_corr/rollout_log_ppl:np.float64(0.26023055279802065) - rollout_corr/log_ppl_diff:np.float64(0.0003702216345118359) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016211537149501964) - rollout_corr/log_ppl_diff_max:np.float64(0.01012013852596283) - rollout_corr/log_ppl_diff_min:np.float64(-0.012066826224327087) - rollout_corr/ppl_ratio:np.float64(1.0003729804884642) - rollout_corr/chi2_token:np.float64(0.0021484598983079195) - rollout_corr/chi2_seq:np.float64(2.6817988054826856) - actor/pg_loss:np.float64(-0.00016311672516167164) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024571236758674786) - actor/ppo_kl:np.float64(-5.575577448624358e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21021737158298492) - perf/mfu/actor:np.float64(0.08564326289574913) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.05369186401367) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008819486945867538 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008819486945867538 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:709.15625 - response_length/max:4610.0 - response_length/min:62.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:709.15625 - response_length_non_aborted/max:4610.0 - response_length_non_aborted/min:62.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.643472731113434e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.46839356422424316) - timing_s/agent_loop/generate_sequences/max:np.float64(26.743163952603936) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.783247412800847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.743163952603936) - timin
(TaskRunner pid=2049544) Training Progress: 74%|███████▍ | 74/100 [57:39<21:41, 50.06s/it]
(TaskRunner pid=2049544) step:75 - global_seqlen/min:19402 - global_seqlen/max:28818 - global_seqlen/minmax_diff:9416 - global_seqlen/balanced_min:23785 - global_seqlen/balanced_max:23996 - global_seqlen/mean:23814.375 - actor/entropy:0.25116294622421265 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135130643844604 - training/rollout_probs_diff_mean:0.0031402192544192076 - training/rollout_probs_diff_std:0.009609016589820385 - training/rollout_actor_probs_pearson_corr:0.9991532564163208 - rollout_corr/rollout_is_mean:1.0000470876693726 - rollout_corr/rollout_is_ratio_fraction_high:8.058602361415979e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4175806174753234e-05 - rollout_corr/rollout_is_max:2.1670117378234863 - rollout_corr/rollout_is_min:0.4087337255477905 - rollout_corr/rollout_is_std:0.03037172183394432 - rollout_corr/rollout_is_eff_sample_size:0.9990784086315643 - rollout_corr/rollout_is_seq_mean:1.0000663995742798 - rollout_corr/rollout_is_seq_std:0.0019097818294540048 - rollout_corr/rollout_is_seq_max:1.0100950002670288 - rollout_corr/rollout_is_seq_min:0.9921054244041443 - rollout_corr/rollout_is_seq_max_deviation:0.010095000267028809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2536835842765868) - rollout_corr/training_log_ppl:np.float64(0.20439182232803432) - rollout_corr/kl:np.float64(0.00044191136634275097) - rollout_corr/k3_kl:np.float64(0.0004956063722261206) - rollout_corr/rollout_ppl:np.float64(1.2531235865317285) - rollout_corr/rollout_log_ppl:np.float64(0.20394990891509224) - rollout_corr/log_ppl_diff:np.float64(0.0004419134129420854) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013838859085808508) - rollout_corr/log_ppl_diff_max:np.float64(0.009681947529315948) - rollout_corr/log_ppl_diff_min:np.float64(-0.005931958556175232) - rollout_corr/ppl_ratio:np.float64(1.0004437156021595) - rollout_corr/chi2_token:np.float64(0.0011102964635938406) - rollout_corr/chi2_seq:np.float64(0.39504921343177557) - actor/pg_loss:np.float64(1.7428305000066757e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018432266847412393) - actor/ppo_kl:np.float64(3.7105337884923983e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16733924113214016) - perf/mfu/actor:np.float64(0.06918613903772251) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.97526931762695) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008511898340657353 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008511898340657353 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:484.73046875 - response_length/max:2220.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:484.73046875 - response_length_non_aborted/max:2220.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016090832650661469 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2748903073370457) - timing_s/agent_loop/generate_sequences/max:np.float64(13.663885425776243) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.775116956734564) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.66388542577624
(TaskRunner pid=2049544) Training Progress: 75%|███████▌ | 75/100 [58:10<18:31, 44.46s/it]
(TaskRunner pid=2049544) step:76 - global_seqlen/min:25932 - global_seqlen/max:42011 - global_seqlen/minmax_diff:16079 - global_seqlen/balanced_min:31987 - global_seqlen/balanced_max:32015 - global_seqlen/mean:32005.0 - actor/entropy:0.35323843359947205 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8143891096115112 - training/rollout_probs_diff_mean:0.003405920695513487 - training/rollout_probs_diff_std:0.009273294359445572 - training/rollout_actor_probs_pearson_corr:0.9994474053382874 - rollout_corr/rollout_is_mean:1.0001150369644165 - rollout_corr/rollout_is_ratio_fraction_high:1.081595564755844e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.081595564755844e-05 - rollout_corr/rollout_is_max:3.5232980251312256 - rollout_corr/rollout_is_min:0.16788354516029358 - rollout_corr/rollout_is_std:0.030408985912799835 - rollout_corr/rollout_is_eff_sample_size:0.9990763858113328 - rollout_corr/rollout_is_seq_mean:1.0001401901245117 - rollout_corr/rollout_is_seq_std:0.0016239474061876535 - rollout_corr/rollout_is_seq_max:1.010636329650879 - rollout_corr/rollout_is_seq_min:0.9931896328926086 - rollout_corr/rollout_is_seq_max_deviation:0.010636329650878906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3853302095085382) - rollout_corr/training_log_ppl:np.float64(0.3024687277356861) - rollout_corr/kl:np.float64(0.0005689695157187202) - rollout_corr/k3_kl:np.float64(0.0005752426737899441) - rollout_corr/rollout_ppl:np.float64(1.38448647223413) - rollout_corr/rollout_log_ppl:np.float64(0.30189975767279975) - rollout_corr/log_ppl_diff:np.float64(0.0005689700628863648) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012543359625851735) - rollout_corr/log_ppl_diff_max:np.float64(0.007834061980247498) - rollout_corr/log_ppl_diff_min:np.float64(-0.006428062915802002) - rollout_corr/ppl_ratio:np.float64(1.0005704772192985) - rollout_corr/chi2_token:np.float64(0.0012044101022183895) - rollout_corr/chi2_seq:np.float64(1.3872064049355686) - actor/pg_loss:np.float64(5.75545709580183e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002793574865336268) - actor/ppo_kl:np.float64(0.00019198946114684645) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3208663538098335) - perf/mfu/actor:np.float64(0.09134199443488283) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.03634643554688) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008177538402378559 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008177538402378559 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:722.3125 - response_length/max:2396.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:722.3125 - response_length_non_aborted/max:2396.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.119371831417084e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.99646220728755) - timing_s/agent_loop/generate_sequences/max:np.float64(16.732603766024113) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.14587117084011) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.732603766024113) - timing_s/agent_loop/slowest
(TaskRunner pid=2049544) Training Progress: 76%|███████▌ | 76/100 [58:46<16:44, 41.87s/it]
(TaskRunner pid=2049544) step:77 - global_seqlen/min:16244 - global_seqlen/max:41465 - global_seqlen/minmax_diff:25221 - global_seqlen/balanced_min:29420 - global_seqlen/balanced_max:29517 - global_seqlen/mean:29492.5 - actor/entropy:0.3045775294303894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2915552258491516 - training/rollout_probs_diff_mean:0.0032992891501635313 - training/rollout_probs_diff_std:0.009267176501452923 - training/rollout_actor_probs_pearson_corr:0.9993472695350647 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:5.8924742916133255e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.140592098236084 - rollout_corr/rollout_is_min:0.5226767659187317 - rollout_corr/rollout_is_std:0.029582349583506584 - rollout_corr/rollout_is_eff_sample_size:0.9991255307618554 - rollout_corr/rollout_is_seq_mean:1.0000417232513428 - rollout_corr/rollout_is_seq_std:0.0017432396998628974 - rollout_corr/rollout_is_seq_max:1.008344054222107 - rollout_corr/rollout_is_seq_min:0.9936603307723999 - rollout_corr/rollout_is_seq_max_deviation:0.008344054222106934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2967352382838726) - rollout_corr/training_log_ppl:np.float64(0.24739519167633262) - rollout_corr/kl:np.float64(0.0005334063558146696) - rollout_corr/k3_kl:np.float64(0.0005630067024355867) - rollout_corr/rollout_ppl:np.float64(1.2960236202925444) - rollout_corr/rollout_log_ppl:np.float64(0.24686178589763585) - rollout_corr/log_ppl_diff:np.float64(0.0005334057786967605) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014061912952456623) - rollout_corr/log_ppl_diff_max:np.float64(0.01065683364868164) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076304227113723755) - rollout_corr/ppl_ratio:np.float64(1.0005355728790164) - rollout_corr/chi2_token:np.float64(0.0011709912214428186) - rollout_corr/chi2_seq:np.float64(0.44916587066836655) - actor/pg_loss:np.float64(2.6725465431809425e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018346831632243266) - actor/ppo_kl:np.float64(9.181251817746983e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16028263419866562) - perf/mfu/actor:np.float64(0.08482457572663438) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.09542083740234) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006819802336394787 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006819802336394787 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:662.921875 - response_length/max:2972.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:662.921875 - response_length_non_aborted/max:2972.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013144128024578094 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8779960200190544) - timing_s/agent_loop/generate_sequences/max:np.float64(18.480581056326628) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.09187524789013) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.480581056326628) - timing_s/agent_loop/slowes
(TaskRunner pid=2049544) Training Progress: 77%|███████▋ | 77/100 [59:24<15:33, 40.57s/it]
(TaskRunner pid=2049544) step:78 - global_seqlen/min:16539 - global_seqlen/max:34942 - global_seqlen/minmax_diff:18403 - global_seqlen/balanced_min:26643 - global_seqlen/balanced_max:26991 - global_seqlen/mean:26772.625 - actor/entropy:0.324239581823349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30065640807151794 - training/rollout_probs_diff_mean:0.003476528450846672 - training/rollout_probs_diff_std:0.009609517641365528 - training/rollout_actor_probs_pearson_corr:0.9993136525154114 - rollout_corr/rollout_is_mean:0.9999270439147949 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.365252956020413e-05 - rollout_corr/rollout_is_max:1.927759051322937 - rollout_corr/rollout_is_min:0.17178574204444885 - rollout_corr/rollout_is_std:0.031049082055687904 - rollout_corr/rollout_is_eff_sample_size:0.9990367639843101 - rollout_corr/rollout_is_seq_mean:0.9997465014457703 - rollout_corr/rollout_is_seq_std:0.0017120024422183633 - rollout_corr/rollout_is_seq_max:1.0067331790924072 - rollout_corr/rollout_is_seq_min:0.9950670599937439 - rollout_corr/rollout_is_seq_max_deviation:0.0067331790924072266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3026048908941448) - rollout_corr/training_log_ppl:np.float64(0.24134253649390303) - rollout_corr/kl:np.float64(0.0007427987794557112) - rollout_corr/k3_kl:np.float64(0.0005388006466660045) - rollout_corr/rollout_ppl:np.float64(1.301613979972899) - rollout_corr/rollout_log_ppl:np.float64(0.24059973818657454) - rollout_corr/log_ppl_diff:np.float64(0.000742798307328485) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014510368491755798) - rollout_corr/log_ppl_diff_max:np.float64(0.0068367719650268555) - rollout_corr/log_ppl_diff_min:np.float64(-0.005617767572402954) - rollout_corr/ppl_ratio:np.float64(1.0007447020616382) - rollout_corr/chi2_token:np.float64(0.0006613847799599171) - rollout_corr/chi2_seq:np.float64(0.6144026103429496) - actor/pg_loss:np.float64(5.164649337530136e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011791538452143868) - actor/ppo_kl:np.float64(-3.060101226282086e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16195594891905785) - perf/mfu/actor:np.float64(0.07693655141257658) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.07821655273438) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012876042164862156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012876042164862156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:572.23828125 - response_length/max:3226.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:572.23828125 - response_length_non_aborted/max:3226.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.397596836090088e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2444690354168415) - timing_s/agent_loop/generate_sequences/max:np.float64(19.064789429306984) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.27938631172583) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.064789429306984) - timing_s/agent_loop/
(TaskRunner pid=2049544) Training Progress: 78%|███████▊ | 78/100 [1:00:02<14:34, 39.74s/it]
(TaskRunner pid=2049544) step:79 - global_seqlen/min:20249 - global_seqlen/max:46165 - global_seqlen/minmax_diff:25916 - global_seqlen/balanced_min:30262 - global_seqlen/balanced_max:30358 - global_seqlen/mean:30336.375 - actor/entropy:0.27736997604370117 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4035380482673645 - training/rollout_probs_diff_mean:0.0030377230141311884 - training/rollout_probs_diff_std:0.009076771326363087 - training/rollout_actor_probs_pearson_corr:0.9993468523025513 - rollout_corr/rollout_is_mean:0.9998963475227356 - rollout_corr/rollout_is_ratio_fraction_high:5.790153409179766e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.211137977312319e-05 - rollout_corr/rollout_is_max:4.029329299926758 - rollout_corr/rollout_is_min:0.07581690698862076 - rollout_corr/rollout_is_std:0.02898186817765236 - rollout_corr/rollout_is_eff_sample_size:0.9991605182489611 - rollout_corr/rollout_is_seq_mean:0.9999415874481201 - rollout_corr/rollout_is_seq_std:0.0017486511496827006 - rollout_corr/rollout_is_seq_max:1.0085891485214233 - rollout_corr/rollout_is_seq_min:0.9941185712814331 - rollout_corr/rollout_is_seq_max_deviation:0.00858914852142334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2839201726019382) - rollout_corr/training_log_ppl:np.float64(0.2279481466684956) - rollout_corr/kl:np.float64(0.0005260294094213691) - rollout_corr/k3_kl:np.float64(0.0005059725925065095) - rollout_corr/rollout_ppl:np.float64(1.2831925540231168) - rollout_corr/rollout_log_ppl:np.float64(0.2274221143743489) - rollout_corr/log_ppl_diff:np.float64(0.0005260322941467166) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012799371324945241) - rollout_corr/log_ppl_diff_max:np.float64(0.008957505226135254) - rollout_corr/log_ppl_diff_min:np.float64(-0.0052579790353775024) - rollout_corr/ppl_ratio:np.float64(1.0005276040174067) - rollout_corr/chi2_token:np.float64(0.0009765762370079756) - rollout_corr/chi2_seq:np.float64(0.7638932466506958) - actor/pg_loss:np.float64(0.00014414661563932896) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017305412563928257) - actor/ppo_kl:np.float64(-2.993323226707645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20632774755358696) - perf/mfu/actor:np.float64(0.08763345813710535) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.01940155029297) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008221293799579144 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008221293799579144 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:674.63671875 - response_length/max:2725.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:674.63671875 - response_length_non_aborted/max:2725.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010620616376399994 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1851273197680712) - timing_s/agent_loop/generate_sequences/max:np.float64(17.865634333342314) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.507378077250905) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.865634333342314) -
(TaskRunner pid=2049544) Training Progress: 79%|███████▉ | 79/100 [1:00:38<13:35, 38.85s/it]
(TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80}
(TaskRunner pid=2049544) validation generation end
(TaskRunner pid=2049544) [prompt] system
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2049544) <reasoning>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) A
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) user
(TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) A: 50 pC
(TaskRunner pid=2049544) B: 56 pC
(TaskRunner pid=2049544) C: 60 pC
(TaskRunner pid=2049544) D: 64 pC
(TaskRunner pid=2049544) Please reason step by step.
(TaskRunner pid=2049544) assistant
(TaskRunner pid=2049544) <think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) [response] <reasoning>
(TaskRunner pid=2049544) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) E = k \frac{q}{r^2}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) We are given:
(TaskRunner pid=2049544) - $ E = 2.0 \, \text{N/C} $
(TaskRunner pid=2049544) - $ r = 50 \, \text{cm} = 0.50 \, \text{m} $
(TaskRunner pid=2049544) - $ k = 8.99 \times 10^9 \, \text{N m}^2/\text{C}^2 $
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) We need to solve for $ q $:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{E r^2}{k}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Substitute the known values:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{0.5}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \text{C}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C):
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \text{C} = 55.6 \, \text{pC}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) The closest answer is **B: 56 pC**.
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) B
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544) [ground_truth] B
(TaskRunner pid=2049544) [score] 1.0
(TaskRunner pid=2049544) [acc] 1.0
(TaskRunner pid=2049544) [pred] B
(TaskRunner pid=2049544) [incorrect_format] 1
(TaskRunner pid=2049544) [feedback]
(TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80
(WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor
(TaskRunner pid=2049544) step:80 - global_seqlen/min:18167 - global_seqlen/max:36780 - global_seqlen/minmax_diff:18613 - global_seqlen/balanced_min:28127 - global_seqlen/balanced_max:28499 - global_seqlen/mean:28202.25 - actor/entropy:0.23095937073230743 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44294559955596924 - training/rollout_probs_diff_mean:0.002918842015787959 - training/rollout_probs_diff_std:0.0094361687079072 - training/rollout_actor_probs_pearson_corr:0.9991447329521179 - rollout_corr/rollout_is_mean:0.9999209046363831 - rollout_corr/rollout_is_ratio_fraction_high:1.9175455236108974e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.9175455236108974e-05 - rollout_corr/rollout_is_max:2.320603370666504 - rollout_corr/rollout_is_min:0.33252665400505066 - rollout_corr/rollout_is_std:0.028494464233517647 - rollout_corr/rollout_is_eff_sample_size:0.999188605215701 - rollout_corr/rollout_is_seq_mean:0.9999192953109741 - rollout_corr/rollout_is_seq_std:0.001614381093531847 - rollout_corr/rollout_is_seq_max:1.0071684122085571 - rollout_corr/rollout_is_seq_min:0.9931057691574097 - rollout_corr/rollout_is_seq_max_deviation:0.007168412208557129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2288768324069679) - rollout_corr/training_log_ppl:np.float64(0.19351409275259357) - rollout_corr/kl:np.float64(0.0005485934744235976) - rollout_corr/k3_kl:np.float64(0.0005014402207166313) - rollout_corr/rollout_ppl:np.float64(1.2281699823215604) - rollout_corr/rollout_log_ppl:np.float64(0.1929654997366015) - rollout_corr/log_ppl_diff:np.float64(0.0005485930159920827) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012800928816432133) - rollout_corr/log_ppl_diff_max:np.float64(0.00843721628189087) - rollout_corr/log_ppl_diff_min:np.float64(-0.006566546857357025) - rollout_corr/ppl_ratio:np.float64(1.0005502616986632) - rollout_corr/chi2_token:np.float64(0.0008651481475681067) - rollout_corr/chi2_seq:np.float64(0.5386446032207459) - actor/pg_loss:np.float64(-2.616771962493658e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001737377756398928) - actor/ppo_kl:np.float64(2.498516215898583e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1959109902381897) - perf/mfu/actor:np.float64(0.08114905418503308) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.12442779541016) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/reward/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10302193213109104) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8416625) - val-aux/sciknoweval/reward/best@16/std:np.float64(
(TaskRunner pid=2049544) Training Progress: 80%|████████ | 80/100 [1:03:17<24:57, 74.90s/it]
(TaskRunner pid=2049544) step:81 - global_seqlen/min:22002 - global_seqlen/max:45026 - global_seqlen/minmax_diff:23024 - global_seqlen/balanced_min:31986 - global_seqlen/balanced_max:32078 - global_seqlen/mean:32047.5 - actor/entropy:0.30386513471603394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135500192642212 - training/rollout_probs_diff_mean:0.003210825379937887 - training/rollout_probs_diff_std:0.00900829304009676 - training/rollout_actor_probs_pearson_corr:0.9993411898612976 - rollout_corr/rollout_is_mean:1.0000028610229492 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.263490493234713e-06 - rollout_corr/rollout_is_max:1.9500842094421387 - rollout_corr/rollout_is_min:0.46151214838027954 - rollout_corr/rollout_is_std:0.029131613671779633 - rollout_corr/rollout_is_eff_sample_size:0.9991520686654056 - rollout_corr/rollout_is_seq_mean:0.9999109506607056 - rollout_corr/rollout_is_seq_std:0.0019725700840353966 - rollout_corr/rollout_is_seq_max:1.0081653594970703 - rollout_corr/rollout_is_seq_min:0.9903681874275208 - rollout_corr/rollout_is_seq_max_deviation:0.009631812572479248 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3244363577105105) - rollout_corr/training_log_ppl:np.float64(0.2665938182544778) - rollout_corr/kl:np.float64(0.0010818528244627146) - rollout_corr/k3_kl:np.float64(0.0008180101519030814) - rollout_corr/rollout_ppl:np.float64(1.3230207040905952) - rollout_corr/rollout_log_ppl:np.float64(0.26551196482614614) - rollout_corr/log_ppl_diff:np.float64(0.0010818534283316694) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018223601437057368) - rollout_corr/log_ppl_diff_max:np.float64(0.01393115520477295) - rollout_corr/log_ppl_diff_min:np.float64(-0.010075211524963379) - rollout_corr/ppl_ratio:np.float64(1.0010864462237805) - rollout_corr/chi2_token:np.float64(0.0015508977230638266) - rollout_corr/chi2_seq:np.float64(1.0036336749326438) - actor/pg_loss:np.float64(7.214734796434641e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002760089199682625) - actor/ppo_kl:np.float64(0.0004848889264348344) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21540894359350204) - perf/mfu/actor:np.float64(0.0899665554810477) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.27264404296875) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004168026614934206 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004168026614934206 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:742.140625 - response_length/max:3366.0 - response_length/min:63.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:742.140625 - response_length_non_aborted/max:3366.0 - response_length_non_aborted/min:63.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.953603148460388e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6883454918861389) - timing_s/agent_loop/generate_sequences/max:np.float64(21.4869614392519) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.608749296545284) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.4869614392519) - timing_s/agent_loop/slowest/tool_c
(TaskRunner pid=2049544) Training Progress: 81%|████████ | 81/100 [1:03:58<20:28, 64.65s/it]
(TaskRunner pid=2049544) step:82 - global_seqlen/min:25371 - global_seqlen/max:44114 - global_seqlen/minmax_diff:18743 - global_seqlen/balanced_min:33520 - global_seqlen/balanced_max:33584 - global_seqlen/mean:33563.875 - actor/entropy:0.2920665442943573 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.913862407207489 - training/rollout_probs_diff_mean:0.0031529199331998825 - training/rollout_probs_diff_std:0.009444354102015495 - training/rollout_actor_probs_pearson_corr:0.9993178248405457 - rollout_corr/rollout_is_mean:0.9999819993972778 - rollout_corr/rollout_is_ratio_fraction_high:2.0209470676491037e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.547131175058894e-05 - rollout_corr/rollout_is_max:2.713327169418335 - rollout_corr/rollout_is_min:0.049278318881988525 - rollout_corr/rollout_is_std:0.02968292124569416 - rollout_corr/rollout_is_eff_sample_size:0.9991195807527394 - rollout_corr/rollout_is_seq_mean:0.9999279379844666 - rollout_corr/rollout_is_seq_std:0.0012865856988355517 - rollout_corr/rollout_is_seq_max:1.004623532295227 - rollout_corr/rollout_is_seq_min:0.9942681193351746 - rollout_corr/rollout_is_seq_max_deviation:0.0057318806648254395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3072700020857155) - rollout_corr/training_log_ppl:np.float64(0.24861305610102136) - rollout_corr/kl:np.float64(0.0007281105056762271) - rollout_corr/k3_kl:np.float64(0.0005667303261134293) - rollout_corr/rollout_ppl:np.float64(1.306269062217325) - rollout_corr/rollout_log_ppl:np.float64(0.2478849448962137) - rollout_corr/log_ppl_diff:np.float64(0.0007281112048076466) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011902832047780976) - rollout_corr/log_ppl_diff_max:np.float64(0.012343257665634155) - rollout_corr/log_ppl_diff_min:np.float64(-0.00298440083861351) - rollout_corr/ppl_ratio:np.float64(1.0007296884432435) - rollout_corr/chi2_token:np.float64(0.0007541878148913383) - rollout_corr/chi2_seq:np.float64(0.7926566984970123) - actor/pg_loss:np.float64(0.0001337927533313632) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021164456336464355) - actor/ppo_kl:np.float64(0.00017838788005519746) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1409779703244567) - perf/mfu/actor:np.float64(0.09575702121449964) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.184974193573) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00978454202413559 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00978454202413559 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:773.15234375 - response_length/max:2703.0 - response_length/min:191.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:773.15234375 - response_length_non_aborted/max:2703.0 - response_length_non_aborted/min:191.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.613321602344513e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2882147133350372) - timing_s/agent_loop/generate_sequences/max:np.float64(18.499180192127824) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.74611435400584) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.499180192127824) - timin
(TaskRunner pid=2049544) Training Progress: 82%|████████▏ | 82/100 [1:04:35<16:55, 56.42s/it]
(TaskRunner pid=2049544) step:83 - global_seqlen/min:19276 - global_seqlen/max:41937 - global_seqlen/minmax_diff:22661 - global_seqlen/balanced_min:29583 - global_seqlen/balanced_max:29645 - global_seqlen/mean:29615.125 - actor/entropy:0.34415289759635925 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5622131824493408 - training/rollout_probs_diff_mean:0.003517036559060216 - training/rollout_probs_diff_std:0.009729686193168163 - training/rollout_actor_probs_pearson_corr:0.9993311762809753 - rollout_corr/rollout_is_mean:0.9999920129776001 - rollout_corr/rollout_is_ratio_fraction_high:1.2048701137246098e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.0121751478873193e-05 - rollout_corr/rollout_is_max:8.23111629486084 - rollout_corr/rollout_is_min:0.2471064180135727 - rollout_corr/rollout_is_std:0.03159145638346672 - rollout_corr/rollout_is_eff_sample_size:0.9990024989924954 - rollout_corr/rollout_is_seq_mean:0.9999005794525146 - rollout_corr/rollout_is_seq_std:0.0016829889500513673 - rollout_corr/rollout_is_seq_max:1.004620909690857 - rollout_corr/rollout_is_seq_min:0.9928184747695923 - rollout_corr/rollout_is_seq_max_deviation:0.007181525230407715 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.372836111113429) - rollout_corr/training_log_ppl:np.float64(0.2916473155346466) - rollout_corr/kl:np.float64(0.0007178986236571205) - rollout_corr/k3_kl:np.float64(0.0006351841151968074) - rollout_corr/rollout_ppl:np.float64(1.3718020445667207) - rollout_corr/rollout_log_ppl:np.float64(0.2909294152777875) - rollout_corr/log_ppl_diff:np.float64(0.0007179002568591386) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014387322589755058) - rollout_corr/log_ppl_diff_max:np.float64(0.009183712303638458) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071634650230407715) - rollout_corr/ppl_ratio:np.float64(1.0007200299296528) - rollout_corr/chi2_token:np.float64(0.0011453672777861357) - rollout_corr/chi2_seq:np.float64(0.5700722692999989) - actor/pg_loss:np.float64(0.00010708440095186234) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003794109481987107) - actor/ppo_kl:np.float64(5.79457945377726e-05) - actor/pg_clipfrac_lower:np.float64(4.787071247847052e-06) - actor/grad_norm:np.float64(0.27837222814559937) - perf/mfu/actor:np.float64(0.08514566987298042) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.15073013305664) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003308121347799897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003308121347799897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:648.41015625 - response_length/max:2853.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:648.41015625 - response_length_non_aborted/max:2853.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.170570850372314e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8073275871574879) - timing_s/agent_loop/generate_sequences/max:np.float64(18.502459589391947) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.3145277952207834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.50245
(TaskRunner pid=2049544) Training Progress: 83%|████████▎ | 83/100 [1:05:12<14:21, 50.66s/it]
(TaskRunner pid=2049544) step:84 - global_seqlen/min:25752 - global_seqlen/max:42945 - global_seqlen/minmax_diff:17193 - global_seqlen/balanced_min:33932 - global_seqlen/balanced_max:34032 - global_seqlen/mean:33969.0 - actor/entropy:0.3499566912651062 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6138913631439209 - training/rollout_probs_diff_mean:0.0033732259180396795 - training/rollout_probs_diff_std:0.009146650321781635 - training/rollout_actor_probs_pearson_corr:0.9994776844978333 - rollout_corr/rollout_is_mean:0.9999618530273438 - rollout_corr/rollout_is_ratio_fraction_high:4.901384272670839e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4506920453859493e-05 - rollout_corr/rollout_is_max:2.404789447784424 - rollout_corr/rollout_is_min:0.2902381718158722 - rollout_corr/rollout_is_std:0.03017878346145153 - rollout_corr/rollout_is_eff_sample_size:0.9990900697549667 - rollout_corr/rollout_is_seq_mean:1.0001087188720703 - rollout_corr/rollout_is_seq_std:0.001655240310356021 - rollout_corr/rollout_is_seq_max:1.0082566738128662 - rollout_corr/rollout_is_seq_min:0.9946045279502869 - rollout_corr/rollout_is_seq_max_deviation:0.008256673812866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3845429411157966) - rollout_corr/training_log_ppl:np.float64(0.29742183088092133) - rollout_corr/kl:np.float64(0.00039445633141355074) - rollout_corr/k3_kl:np.float64(0.0005463047522766828) - rollout_corr/rollout_ppl:np.float64(1.3839370189234614) - rollout_corr/rollout_log_ppl:np.float64(0.29702737282786984) - rollout_corr/log_ppl_diff:np.float64(0.00039445805305149406) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012936670245835558) - rollout_corr/log_ppl_diff_max:np.float64(0.00665166974067688) - rollout_corr/log_ppl_diff_min:np.float64(-0.005310043692588806) - rollout_corr/ppl_ratio:np.float64(1.000396022805944) - rollout_corr/chi2_token:np.float64(0.0013724297750741243) - rollout_corr/chi2_seq:np.float64(1.7039004366379231) - actor/pg_loss:np.float64(0.00012169475667178631) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021371153513882746) - actor/ppo_kl:np.float64(1.5052716766383867e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2066413126885891) - perf/mfu/actor:np.float64(0.09535977782956703) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.16190719604492) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017204470932483673 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017204470932483673 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:796.96875 - response_length/max:3594.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:796.96875 - response_length_non_aborted/max:3594.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7237460017204285e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9937344156205654) - timing_s/agent_loop/generate_sequences/max:np.float64(23.47761689312756) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.72609610089421) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.47761689312756) - timing_s/agent_
(TaskRunner pid=2049544) Training Progress: 84%|████████▍ | 84/100 [1:05:55<12:52, 48.31s/it]
(TaskRunner pid=2049544) step:85 - global_seqlen/min:20755 - global_seqlen/max:54433 - global_seqlen/minmax_diff:33678 - global_seqlen/balanced_min:33282 - global_seqlen/balanced_max:33493 - global_seqlen/mean:33418.125 - actor/entropy:0.29106423258781433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967511296272278 - training/rollout_probs_diff_mean:0.003082117298617959 - training/rollout_probs_diff_std:0.009240406565368176 - training/rollout_actor_probs_pearson_corr:0.9993290901184082 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:4.992785306967562e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.9971141227870248e-05 - rollout_corr/rollout_is_max:2.3519351482391357 - rollout_corr/rollout_is_min:0.26859748363494873 - rollout_corr/rollout_is_std:0.029412612318992615 - rollout_corr/rollout_is_eff_sample_size:0.9991358839466532 - rollout_corr/rollout_is_seq_mean:0.9999825954437256 - rollout_corr/rollout_is_seq_std:0.0015941840829327703 - rollout_corr/rollout_is_seq_max:1.0068665742874146 - rollout_corr/rollout_is_seq_min:0.9917056560516357 - rollout_corr/rollout_is_seq_max_deviation:0.008294343948364258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2718443693593144) - rollout_corr/training_log_ppl:np.float64(0.22443578000093112) - rollout_corr/kl:np.float64(0.0005144908149414817) - rollout_corr/k3_kl:np.float64(0.0004981835849378058) - rollout_corr/rollout_ppl:np.float64(1.2711914060637355) - rollout_corr/rollout_log_ppl:np.float64(0.22392128901265096) - rollout_corr/log_ppl_diff:np.float64(0.0005144909882801585) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011518053579493426) - rollout_corr/log_ppl_diff_max:np.float64(0.009792208671569824) - rollout_corr/log_ppl_diff_min:np.float64(-0.005565732717514038) - rollout_corr/ppl_ratio:np.float64(1.000515874940902) - rollout_corr/chi2_token:np.float64(0.000967925414443016) - rollout_corr/chi2_seq:np.float64(1.0396802069153637) - actor/pg_loss:np.float64(-0.0001277067931368947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000293373741897085) - actor/ppo_kl:np.float64(1.70920517623685e-05) - actor/pg_clipfrac_lower:np.float64(5.778476406703703e-06) - actor/grad_norm:np.float64(0.20695016533136368) - perf/mfu/actor:np.float64(0.09336689611238912) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0907974243164) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014600777067244053 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014600777067244053 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:782.37890625 - response_length/max:4261.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:782.37890625 - response_length_non_aborted/max:4261.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.2804364562034607e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7515706215053797) - timing_s/agent_loop/generate_sequences/max:np.float64(26.178023107349873) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.071110863718786) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.178023107
(TaskRunner pid=2049544) Training Progress: 85%|████████▌ | 85/100 [1:06:41<11:51, 47.43s/it]
(TaskRunner pid=2049544) step:86 - global_seqlen/min:22104 - global_seqlen/max:49662 - global_seqlen/minmax_diff:27558 - global_seqlen/balanced_min:36725 - global_seqlen/balanced_max:36775 - global_seqlen/mean:36762.375 - actor/entropy:0.3210241496562958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24132895469665527 - training/rollout_probs_diff_mean:0.003310859901830554 - training/rollout_probs_diff_std:0.009120251052081585 - training/rollout_actor_probs_pearson_corr:0.9993811845779419 - rollout_corr/rollout_is_mean:0.9998584389686584 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2271813577390276e-05 - rollout_corr/rollout_is_max:1.9132159948349 - rollout_corr/rollout_is_min:0.41626274585723877 - rollout_corr/rollout_is_std:0.029449068009853363 - rollout_corr/rollout_is_eff_sample_size:0.9991331468769348 - rollout_corr/rollout_is_seq_mean:0.9998147487640381 - rollout_corr/rollout_is_seq_std:0.0013719926355406642 - rollout_corr/rollout_is_seq_max:1.0041395425796509 - rollout_corr/rollout_is_seq_min:0.9918246865272522 - rollout_corr/rollout_is_seq_max_deviation:0.008175313472747803 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.318041059654206) - rollout_corr/training_log_ppl:np.float64(0.26210865678149275) - rollout_corr/kl:np.float64(0.0005536363921763154) - rollout_corr/k3_kl:np.float64(0.0004948169941201286) - rollout_corr/rollout_ppl:np.float64(1.317284729797393) - rollout_corr/rollout_log_ppl:np.float64(0.2615550198825076) - rollout_corr/log_ppl_diff:np.float64(0.0005536368989851326) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010979093494825065) - rollout_corr/log_ppl_diff_max:np.float64(0.0066015273332595825) - rollout_corr/log_ppl_diff_min:np.float64(-0.002927526831626892) - rollout_corr/ppl_ratio:np.float64(1.0005547057371587) - rollout_corr/chi2_token:np.float64(0.0008581595029681921) - rollout_corr/chi2_seq:np.float64(0.48298407695256174) - actor/pg_loss:np.float64(0.00010274478700011969) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012036708881169034) - actor/ppo_kl:np.float64(-9.579434273687681e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1470967587083578) - perf/mfu/actor:np.float64(0.1030748564778111) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.2135009765625) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007343573961406946 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007343573961406946 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:876.94921875 - response_length/max:3278.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:876.94921875 - response_length_non_aborted/max:3278.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010039471089839935 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4529647156596184) - timing_s/agent_loop/generate_sequences/max:np.float64(22.549825344234705) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.41551528852142) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.549825344234705) - timing_s/agent_loop/slow
(TaskRunner pid=2049544) Training Progress: 86%|████████▌ | 86/100 [1:07:23<10:42, 45.88s/it]
(TaskRunner pid=2049544) step:87 - global_seqlen/min:29440 - global_seqlen/max:47768 - global_seqlen/minmax_diff:18328 - global_seqlen/balanced_min:35517 - global_seqlen/balanced_max:36384 - global_seqlen/mean:35692.75 - actor/entropy:0.3466551899909973 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292303800582886 - training/rollout_probs_diff_mean:0.003489995375275612 - training/rollout_probs_diff_std:0.00928904302418232 - training/rollout_actor_probs_pearson_corr:0.9994551539421082 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.662482751882635e-06 - rollout_corr/rollout_is_max:1.702067494392395 - rollout_corr/rollout_is_min:0.26140108704566956 - rollout_corr/rollout_is_std:0.03009967692196369 - rollout_corr/rollout_is_eff_sample_size:0.9990948294754013 - rollout_corr/rollout_is_seq_mean:1.0000789165496826 - rollout_corr/rollout_is_seq_std:0.001175415120087564 - rollout_corr/rollout_is_seq_max:1.0036451816558838 - rollout_corr/rollout_is_seq_min:0.9961031079292297 - rollout_corr/rollout_is_seq_max_deviation:0.0038968920707702637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.384202916175127) - rollout_corr/training_log_ppl:np.float64(0.2980515792296501) - rollout_corr/kl:np.float64(0.00042411336519165843) - rollout_corr/k3_kl:np.float64(0.00048466515880818406) - rollout_corr/rollout_ppl:np.float64(1.3836216540075839) - rollout_corr/rollout_log_ppl:np.float64(0.2976274649699917) - rollout_corr/log_ppl_diff:np.float64(0.00042411425965838134) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010522096417844296) - rollout_corr/log_ppl_diff_max:np.float64(0.004691094160079956) - rollout_corr/log_ppl_diff_min:np.float64(-0.005741387605667114) - rollout_corr/ppl_ratio:np.float64(1.0004250849597156) - rollout_corr/chi2_token:np.float64(0.001095710089430213) - rollout_corr/chi2_seq:np.float64(1.8486225868109614) - actor/pg_loss:np.float64(7.82626448199153e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016853076499501185) - actor/ppo_kl:np.float64(3.361720549453773e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18693841621279716) - perf/mfu/actor:np.float64(0.0996124497047341) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.18145751953125) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006155060138553381 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006155060138553381 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:837.8046875 - response_length/max:4550.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:837.8046875 - response_length_non_aborted/max:4550.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001092962920665741 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4342339634895325) - timing_s/agent_loop/generate_sequences/max:np.float64(28.061727035790682) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.031970594347513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.061727035790682) - timing_s/agent_loop/slowest/tool
(TaskRunner pid=2049544) Training Progress: 87%|████████▋ | 87/100 [1:08:10<10:02, 46.32s/it]
(TaskRunner pid=2049544) step:88 - global_seqlen/min:22456 - global_seqlen/max:46812 - global_seqlen/minmax_diff:24356 - global_seqlen/balanced_min:35428 - global_seqlen/balanced_max:35596 - global_seqlen/mean:35553.625 - actor/entropy:0.3236216604709625 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21511626243591309 - training/rollout_probs_diff_mean:0.0032852201256901026 - training/rollout_probs_diff_std:0.009024620987474918 - training/rollout_actor_probs_pearson_corr:0.999454140663147 - rollout_corr/rollout_is_mean:1.0000885725021362 - rollout_corr/rollout_is_ratio_fraction_high:4.604963123711059e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.604963123711059e-06 - rollout_corr/rollout_is_max:2.0104920864105225 - rollout_corr/rollout_is_min:0.23549965023994446 - rollout_corr/rollout_is_std:0.029910963028669357 - rollout_corr/rollout_is_eff_sample_size:0.999106252989479 - rollout_corr/rollout_is_seq_mean:1.0000383853912354 - rollout_corr/rollout_is_seq_std:0.0014727648813277483 - rollout_corr/rollout_is_seq_max:1.0041821002960205 - rollout_corr/rollout_is_seq_min:0.9893348813056946 - rollout_corr/rollout_is_seq_max_deviation:0.01066511869430542 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.338182927109301) - rollout_corr/training_log_ppl:np.float64(0.2705517132417299) - rollout_corr/kl:np.float64(0.0005242268059881816) - rollout_corr/k3_kl:np.float64(0.000508854052498009) - rollout_corr/rollout_ppl:np.float64(1.3374784495681524) - rollout_corr/rollout_log_ppl:np.float64(0.27002748603990767) - rollout_corr/log_ppl_diff:np.float64(0.0005242272018222138) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011741442867787555) - rollout_corr/log_ppl_diff_max:np.float64(0.013874843716621399) - rollout_corr/log_ppl_diff_min:np.float64(-0.0045333802700042725) - rollout_corr/ppl_ratio:np.float64(1.0005257281009108) - rollout_corr/chi2_token:np.float64(0.0009958110749721527) - rollout_corr/chi2_seq:np.float64(1.3633345067501068) - actor/pg_loss:np.float64(1.946359407156706e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016658261313295952) - actor/ppo_kl:np.float64(7.023494554836418e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13375208340585232) - perf/mfu/actor:np.float64(0.09886611268499279) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.15729522705078) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02385486103594303 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02385486103594303 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:848.26953125 - response_length/max:4442.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:848.26953125 - response_length_non_aborted/max:4442.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.980984032154083e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.990953391417861) - timing_s/agent_loop/generate_sequences/max:np.float64(25.68886986002326) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.6535395380997215) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.68886986002326) - timing
(TaskRunner pid=2049544) Training Progress: 88%|████████▊ | 88/100 [1:08:55<09:11, 45.95s/it]
(TaskRunner pid=2049544) step:89 - global_seqlen/min:23526 - global_seqlen/max:43993 - global_seqlen/minmax_diff:20467 - global_seqlen/balanced_min:32615 - global_seqlen/balanced_max:32696 - global_seqlen/mean:32672.25 - actor/entropy:0.3472384810447693 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7165358662605286 - training/rollout_probs_diff_mean:0.003400100627914071 - training/rollout_probs_diff_std:0.009368252009153366 - training/rollout_actor_probs_pearson_corr:0.9993821382522583 - rollout_corr/rollout_is_mean:0.9999688267707825 - rollout_corr/rollout_is_ratio_fraction_high:5.2063269322388805e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.123795977444388e-05 - rollout_corr/rollout_is_max:2.0695598125457764 - rollout_corr/rollout_is_min:0.2369716465473175 - rollout_corr/rollout_is_std:0.03049706295132637 - rollout_corr/rollout_is_eff_sample_size:0.9990706743631297 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0016654012724757195 - rollout_corr/rollout_is_seq_max:1.007972240447998 - rollout_corr/rollout_is_seq_min:0.9951695799827576 - rollout_corr/rollout_is_seq_max_deviation:0.007972240447998047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3569685937836766) - rollout_corr/training_log_ppl:np.float64(0.27856632304610685) - rollout_corr/kl:np.float64(0.0005558686453759343) - rollout_corr/k3_kl:np.float64(0.0005486493795388014) - rollout_corr/rollout_ppl:np.float64(1.3561613205820322) - rollout_corr/rollout_log_ppl:np.float64(0.27801045119122136) - rollout_corr/log_ppl_diff:np.float64(0.0005558718548854813) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011855626216856763) - rollout_corr/log_ppl_diff_max:np.float64(0.005568467080593109) - rollout_corr/log_ppl_diff_min:np.float64(-0.006579384207725525) - rollout_corr/ppl_ratio:np.float64(1.0005573213566095) - rollout_corr/chi2_token:np.float64(0.0010650090407580137) - rollout_corr/chi2_seq:np.float64(0.8179540059063584) - actor/pg_loss:np.float64(3.63390427082777e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022707523396547913) - actor/ppo_kl:np.float64(6.390776004217003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3162117004394531) - perf/mfu/actor:np.float64(0.09205693492850642) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.16839599609375) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01907663233578205 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01907663233578205 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:750.2890625 - response_length/max:3252.0 - response_length/min:118.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:750.2890625 - response_length_non_aborted/max:3252.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003261677920818329 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3473277054727077) - timing_s/agent_loop/generate_sequences/max:np.float64(20.700606567785144) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.879736434209917) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.700606567785144) - timing_s
(TaskRunner pid=2049544) Training Progress: 89%|████████▉ | 89/100 [1:09:36<08:07, 44.35s/it]
(TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90}
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) validation generation end
(TaskRunner pid=2049544) [prompt] system
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2049544) <reasoning>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) A
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) user
(TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) A: 50 pC
(TaskRunner pid=2049544) B: 56 pC
(TaskRunner pid=2049544) C: 60 pC
(TaskRunner pid=2049544) D: 64 pC
(TaskRunner pid=2049544) Please reason step by step.
(TaskRunner pid=2049544) assistant
(TaskRunner pid=2049544) <think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) [response] <reasoning>
(TaskRunner pid=2049544) We are given:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \text{N/C} $
(TaskRunner pid=2049544) - Distance $ r = 50 \, \text{cm} = 0.50 \, \text{m} $
(TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \text{N m}^2/\text{C}^2 $
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) We are to find the charge $ q $ using the formula:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) E = k \frac{q}{r^2}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Rearranging to solve for $ q $:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{E r^2}{k}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Substitute the known values:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \text{C}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C):
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \text{C} = 55.6 \, \text{pC}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) The closest answer is **B: 56 pC**.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) B
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544) [ground_truth] B
(TaskRunner pid=2049544) [score] 1.0
(TaskRunner pid=2049544) [acc] 1.0
(TaskRunner pid=2049544) [pred] B
(TaskRunner pid=2049544) [incorrect_format] 1
(TaskRunner pid=2049544) [feedback]
(TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90
(WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor
(TaskRunner pid=2049544) step:90 - global_seqlen/min:22189 - global_seqlen/max:43070 - global_seqlen/minmax_diff:20881 - global_seqlen/balanced_min:29230 - global_seqlen/balanced_max:29298 - global_seqlen/mean:29287.75 - actor/entropy:0.3101608455181122 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3454935550689697 - training/rollout_probs_diff_mean:0.0033272632863372564 - training/rollout_probs_diff_std:0.009299682453274727 - training/rollout_actor_probs_pearson_corr:0.999302327632904 - rollout_corr/rollout_is_mean:0.9999826550483704 - rollout_corr/rollout_is_ratio_fraction_high:1.2050661098328419e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2050661098328419e-05 - rollout_corr/rollout_is_max:2.8188319206237793 - rollout_corr/rollout_is_min:0.3406546115875244 - rollout_corr/rollout_is_std:0.03000447526574135 - rollout_corr/rollout_is_eff_sample_size:0.9991005411997865 - rollout_corr/rollout_is_seq_mean:1.0000338554382324 - rollout_corr/rollout_is_seq_std:0.0013818239094689488 - rollout_corr/rollout_is_seq_max:1.0056010484695435 - rollout_corr/rollout_is_seq_min:0.9940690398216248 - rollout_corr/rollout_is_seq_max_deviation:0.005930960178375244 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3156746202148497) - rollout_corr/training_log_ppl:np.float64(0.2531014566193335) - rollout_corr/kl:np.float64(0.0005285690323812986) - rollout_corr/k3_kl:np.float64(0.0004731722787880699) - rollout_corr/rollout_ppl:np.float64(1.3148961896076798) - rollout_corr/rollout_log_ppl:np.float64(0.25257288754801266) - rollout_corr/log_ppl_diff:np.float64(0.0005285690713208169) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010851562838070095) - rollout_corr/log_ppl_diff_max:np.float64(0.006567090749740601) - rollout_corr/log_ppl_diff_min:np.float64(-0.00448918342590332) - rollout_corr/ppl_ratio:np.float64(1.0005297421012074) - rollout_corr/chi2_token:np.float64(0.0008521552663296461) - rollout_corr/chi2_seq:np.float64(0.22603251412510872) - actor/pg_loss:np.float64(0.00011028861626982689) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012377322673273738) - actor/ppo_kl:np.float64(0.00010799788028137414) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14029540959745646) - perf/mfu/actor:np.float64(0.08421706394502831) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.20226669311523) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6625) - val-aux/sciknoweval/reward/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10800278634919543) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8915625) - val-aux/sciknoweval/reward/best@16/std:np.flo
(TaskRunner pid=2049544) Training Progress: 90%|█████████ | 90/100 [1:12:12<12:58, 77.88s/it]
(TaskRunner pid=2049544) step:91 - global_seqlen/min:21497 - global_seqlen/max:45454 - global_seqlen/minmax_diff:23957 - global_seqlen/balanced_min:32904 - global_seqlen/balanced_max:33139 - global_seqlen/mean:33002.875 - actor/entropy:0.26041534543037415 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34585317969322205 - training/rollout_probs_diff_mean:0.002946822438389063 - training/rollout_probs_diff_std:0.008950787596404552 - training/rollout_actor_probs_pearson_corr:0.9992419481277466 - rollout_corr/rollout_is_mean:0.9999063611030579 - rollout_corr/rollout_is_ratio_fraction_high:5.014869202568661e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.5044607607705984e-05 - rollout_corr/rollout_is_max:5.096095561981201 - rollout_corr/rollout_is_min:0.37255313992500305 - rollout_corr/rollout_is_std:0.028246553614735603 - rollout_corr/rollout_is_eff_sample_size:0.9992028873298154 - rollout_corr/rollout_is_seq_mean:1.0000227689743042 - rollout_corr/rollout_is_seq_std:0.001477584009990096 - rollout_corr/rollout_is_seq_max:1.0067633390426636 - rollout_corr/rollout_is_seq_min:0.995822548866272 - rollout_corr/rollout_is_seq_max_deviation:0.006763339042663574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2352091469801962) - rollout_corr/training_log_ppl:np.float64(0.20221140759531409) - rollout_corr/kl:np.float64(0.00045496249386900445) - rollout_corr/k3_kl:np.float64(0.00047493476955651204) - rollout_corr/rollout_ppl:np.float64(1.2346553695388138) - rollout_corr/rollout_log_ppl:np.float64(0.2017564445413882) - rollout_corr/log_ppl_diff:np.float64(0.0004549630539258942) - rollout_corr/log_ppl_abs_diff:np.float64(0.001167977330624126) - rollout_corr/log_ppl_diff_max:np.float64(0.009916573762893677) - rollout_corr/log_ppl_diff_min:np.float64(-0.006213173270225525) - rollout_corr/ppl_ratio:np.float64(1.0004563292022794) - rollout_corr/chi2_token:np.float64(0.0009643365629017353) - rollout_corr/chi2_seq:np.float64(0.6764775053597987) - actor/pg_loss:np.float64(5.630298983305693e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001349058953792337) - actor/ppo_kl:np.float64(3.4980036296872186e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12929855473339558) - perf/mfu/actor:np.float64(0.09015558565254489) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.73992156982422) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012762214988470078 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012762214988470078 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:778.93359375 - response_length/max:5011.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:778.93359375 - response_length_non_aborted/max:5011.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.5048614740371704e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9828272927552462) - timing_s/agent_loop/generate_sequences/max:np.float64(30.033628037199378) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.231723692668311) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(30.033628037199378)
(TaskRunner pid=2049544) Training Progress: 91%|█████████ | 91/100 [1:13:02<10:24, 69.39s/it]
(TaskRunner pid=2049544) step:92 - global_seqlen/min:27995 - global_seqlen/max:54733 - global_seqlen/minmax_diff:26738 - global_seqlen/balanced_min:38799 - global_seqlen/balanced_max:39948 - global_seqlen/mean:39032.875 - actor/entropy:0.365276038646698 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4553599953651428 - training/rollout_probs_diff_mean:0.0034760674461722374 - training/rollout_probs_diff_std:0.009343450888991356 - training/rollout_actor_probs_pearson_corr:0.999370813369751 - rollout_corr/rollout_is_mean:0.9999127984046936 - rollout_corr/rollout_is_ratio_fraction_high:8.297963177028578e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.0744908397318795e-05 - rollout_corr/rollout_is_max:3.3402388095855713 - rollout_corr/rollout_is_min:0.3921363651752472 - rollout_corr/rollout_is_std:0.03065885230898857 - rollout_corr/rollout_is_eff_sample_size:0.999060798471362 - rollout_corr/rollout_is_seq_mean:0.9999949932098389 - rollout_corr/rollout_is_seq_std:0.001438560662791133 - rollout_corr/rollout_is_seq_max:1.005057454109192 - rollout_corr/rollout_is_seq_min:0.9950283765792847 - rollout_corr/rollout_is_seq_max_deviation:0.0050574541091918945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4253517426550388) - rollout_corr/training_log_ppl:np.float64(0.3268485037915525) - rollout_corr/kl:np.float64(0.0007143817989492263) - rollout_corr/k3_kl:np.float64(0.0007667658379375553) - rollout_corr/rollout_ppl:np.float64(1.4242633627727628) - rollout_corr/rollout_log_ppl:np.float64(0.3261341213074047) - rollout_corr/log_ppl_diff:np.float64(0.0007143824841477908) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013714263404835947) - rollout_corr/log_ppl_diff_max:np.float64(0.013571739196777344) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037727057933807373) - rollout_corr/ppl_ratio:np.float64(1.000716715119779) - rollout_corr/chi2_token:np.float64(0.001422470435500145) - rollout_corr/chi2_seq:np.float64(0.832664096262306) - actor/pg_loss:np.float64(2.3467233404517174e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003434544945548623) - actor/ppo_kl:np.float64(0.00016380211024369373) - actor/pg_clipfrac_lower:np.float64(2.0755844616360264e-06) - actor/grad_norm:np.float64(0.215729471296072) - perf/mfu/actor:np.float64(0.10647986423451218) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.4896354675293) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011413848027586937 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011413848027586937 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:941.49609375 - response_length/max:5473.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:941.49609375 - response_length_non_aborted/max:5473.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.439482629299164e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7557445913553238) - timing_s/agent_loop/generate_sequences/max:np.float64(32.87127552181482) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.810325469457894) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(32.87127552
(TaskRunner pid=2049544) Training Progress: 92%|█████████▏| 92/100 [1:13:54<08:35, 64.39s/it]
(TaskRunner pid=2049544) step:93 - global_seqlen/min:17843 - global_seqlen/max:51915 - global_seqlen/minmax_diff:34072 - global_seqlen/balanced_min:30929 - global_seqlen/balanced_max:31113 - global_seqlen/mean:31065.0 - actor/entropy:0.4133561849594116 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311214327812195 - training/rollout_probs_diff_mean:0.0036772009916603565 - training/rollout_probs_diff_std:0.009407786652445793 - training/rollout_actor_probs_pearson_corr:0.9994404911994934 - rollout_corr/rollout_is_mean:0.9999762177467346 - rollout_corr/rollout_is_ratio_fraction_high:5.474292720464291e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6422878616140224e-05 - rollout_corr/rollout_is_max:2.111367702484131 - rollout_corr/rollout_is_min:0.21797895431518555 - rollout_corr/rollout_is_std:0.031457215547561646 - rollout_corr/rollout_is_eff_sample_size:0.9990114219410976 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0018004179000854492 - rollout_corr/rollout_is_seq_max:1.007155179977417 - rollout_corr/rollout_is_seq_min:0.9935778975486755 - rollout_corr/rollout_is_seq_max_deviation:0.007155179977416992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4254543129354715) - rollout_corr/training_log_ppl:np.float64(0.3189181691413978) - rollout_corr/kl:np.float64(0.0006560000657124254) - rollout_corr/k3_kl:np.float64(0.0006174342987037562) - rollout_corr/rollout_ppl:np.float64(1.4244641666300595) - rollout_corr/rollout_log_ppl:np.float64(0.31826216776971705) - rollout_corr/log_ppl_diff:np.float64(0.0006560013716807589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014309252292150632) - rollout_corr/log_ppl_diff_max:np.float64(0.00973115861415863) - rollout_corr/log_ppl_diff_min:np.float64(-0.004590287804603577) - rollout_corr/ppl_ratio:np.float64(1.0006579444743693) - rollout_corr/chi2_token:np.float64(0.0011531137861311436) - rollout_corr/chi2_seq:np.float64(0.8776039613876492) - actor/pg_loss:np.float64(0.00012877536937594414) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000180150422920633) - actor/ppo_kl:np.float64(9.64217530006195e-05) - actor/pg_clipfrac_lower:np.float64(3.570612534531392e-06) - actor/grad_norm:np.float64(0.18000993877649307) - perf/mfu/actor:np.float64(0.08642174822282235) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.50779724121094) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016780760139226913 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016780760139226913 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:713.5625 - response_length/max:4647.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:713.5625 - response_length_non_aborted/max:4647.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010390952229499817 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7623797114938498) - timing_s/agent_loop/generate_sequences/max:np.float64(27.237358892336488) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.646888445196964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.237358892336488)
(TaskRunner pid=2049544) Training Progress: 93%|█████████▎| 93/100 [1:14:41<06:53, 59.12s/it]
(TaskRunner pid=2049544) step:94 - global_seqlen/min:22853 - global_seqlen/max:49175 - global_seqlen/minmax_diff:26322 - global_seqlen/balanced_min:33215 - global_seqlen/balanced_max:33359 - global_seqlen/mean:33330.0 - actor/entropy:0.36058807373046875 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8867406845092773 - training/rollout_probs_diff_mean:0.003440327476710081 - training/rollout_probs_diff_std:0.009632454253733158 - training/rollout_actor_probs_pearson_corr:0.9993698000907898 - rollout_corr/rollout_is_mean:0.9999211430549622 - rollout_corr/rollout_is_ratio_fraction_high:1.0051463505078573e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.523158713709563e-05 - rollout_corr/rollout_is_max:2.117000102996826 - rollout_corr/rollout_is_min:0.0013813151745125651 - rollout_corr/rollout_is_std:0.030682172626256943 - rollout_corr/rollout_is_eff_sample_size:0.9990594896328889 - rollout_corr/rollout_is_seq_mean:0.9999591112136841 - rollout_corr/rollout_is_seq_std:0.0020710721146315336 - rollout_corr/rollout_is_seq_max:1.011606216430664 - rollout_corr/rollout_is_seq_min:0.9853842854499817 - rollout_corr/rollout_is_seq_max_deviation:0.01461571455001831 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3442378360778093) - rollout_corr/training_log_ppl:np.float64(0.2689030548208393) - rollout_corr/kl:np.float64(0.0007270200237998736) - rollout_corr/k3_kl:np.float64(0.0005578451756775848) - rollout_corr/rollout_ppl:np.float64(1.3432344142347574) - rollout_corr/rollout_log_ppl:np.float64(0.26817603285599034) - rollout_corr/log_ppl_diff:np.float64(0.0007270219648489729) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013523521920433268) - rollout_corr/log_ppl_diff_max:np.float64(0.018366605043411255) - rollout_corr/log_ppl_diff_min:np.float64(-0.00493505597114563) - rollout_corr/ppl_ratio:np.float64(1.0007292504888028) - rollout_corr/chi2_token:np.float64(0.0007120152004063129) - rollout_corr/chi2_seq:np.float64(0.3402676989790052) - actor/pg_loss:np.float64(-4.822376649826765e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.676928118551587e-05) - actor/ppo_kl:np.float64(0.00014517965372551345) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16034816205501556) - perf/mfu/actor:np.float64(0.09308032359557787) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.54820251464844) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0465502105653286 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0465502105653286 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:777.25 - response_length/max:3614.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:777.25 - response_length_non_aborted/max:3614.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001496635377407074 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8505959454923868) - timing_s/agent_loop/generate_sequences/max:np.float64(23.250541422516108) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.11456365948834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.250541422516108) - timing_s/agent_loop/s
(TaskRunner pid=2049544) Training Progress: 94%|█████████▍| 94/100 [1:15:24<05:25, 54.21s/it]
(TaskRunner pid=2049544) step:95 - global_seqlen/min:27066 - global_seqlen/max:58256 - global_seqlen/minmax_diff:31190 - global_seqlen/balanced_min:37178 - global_seqlen/balanced_max:37249 - global_seqlen/mean:37221.375 - actor/entropy:0.3698161244392395 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553989887237549 - training/rollout_probs_diff_mean:0.0034687381703406572 - training/rollout_probs_diff_std:0.009108307771384716 - training/rollout_actor_probs_pearson_corr:0.999431848526001 - rollout_corr/rollout_is_mean:0.9999311566352844 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3214753380452748e-05 - rollout_corr/rollout_is_max:1.8554537296295166 - rollout_corr/rollout_is_min:0.4261438846588135 - rollout_corr/rollout_is_std:0.030212340876460075 - rollout_corr/rollout_is_eff_sample_size:0.9990878089036473 - rollout_corr/rollout_is_seq_mean:0.9999582767486572 - rollout_corr/rollout_is_seq_std:0.0013638396048918366 - rollout_corr/rollout_is_seq_max:1.0052945613861084 - rollout_corr/rollout_is_seq_min:0.9958161115646362 - rollout_corr/rollout_is_seq_max_deviation:0.0052945613861083984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3786341231316328) - rollout_corr/training_log_ppl:np.float64(0.30179976880026516) - rollout_corr/kl:np.float64(0.0004993855567150263) - rollout_corr/k3_kl:np.float64(0.0005308137937447555) - rollout_corr/rollout_ppl:np.float64(1.377938257995993) - rollout_corr/rollout_log_ppl:np.float64(0.30130038285278715) - rollout_corr/log_ppl_diff:np.float64(0.0004993859474780038) - rollout_corr/log_ppl_abs_diff:np.float64(0.001156414728029631) - rollout_corr/log_ppl_diff_max:np.float64(0.0048898980021476746) - rollout_corr/log_ppl_diff_min:np.float64(-0.00559312105178833) - rollout_corr/ppl_ratio:np.float64(1.0005006100982428) - rollout_corr/chi2_token:np.float64(0.001144631765782833) - rollout_corr/chi2_seq:np.float64(0.9150484425481409) - actor/pg_loss:np.float64(-2.8662499971687794e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000206823247367538) - actor/ppo_kl:np.float64(-2.0183409755958337e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1875382363796234) - perf/mfu/actor:np.float64(0.10496417728509784) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.40451049804688) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.021443139761686325 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.021443139761686325 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:886.79296875 - response_length/max:3209.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:886.79296875 - response_length_non_aborted/max:3209.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.542517364025116e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.471508091315627) - timing_s/agent_loop/generate_sequences/max:np.float64(22.023407490924) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.383664465822221) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.023407490924) - timing_s/agent_loop/slowest/tool_call
(TaskRunner pid=2049544) Training Progress: 95%|█████████▌| 95/100 [1:16:06<04:12, 50.43s/it]
(TaskRunner pid=2049544) step:96 - global_seqlen/min:20271 - global_seqlen/max:71208 - global_seqlen/minmax_diff:50937 - global_seqlen/balanced_min:37576 - global_seqlen/balanced_max:37647 - global_seqlen/mean:37613.5 - actor/entropy:0.334806889295578 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31711164116859436 - training/rollout_probs_diff_mean:0.003466662485152483 - training/rollout_probs_diff_std:0.009574675001204014 - training/rollout_actor_probs_pearson_corr:0.9992694854736328 - rollout_corr/rollout_is_mean:1.0000592470169067 - rollout_corr/rollout_is_ratio_fraction_high:8.720068763068411e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.720068763068411e-06 - rollout_corr/rollout_is_max:2.086963415145874 - rollout_corr/rollout_is_min:0.4454006552696228 - rollout_corr/rollout_is_std:0.030977971851825714 - rollout_corr/rollout_is_eff_sample_size:0.9990414042159874 - rollout_corr/rollout_is_seq_mean:1.0001949071884155 - rollout_corr/rollout_is_seq_std:0.0014157574623823166 - rollout_corr/rollout_is_seq_max:1.0063108205795288 - rollout_corr/rollout_is_seq_min:0.9960592985153198 - rollout_corr/rollout_is_seq_max_deviation:0.006310820579528809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3516691238619387) - rollout_corr/training_log_ppl:np.float64(0.27739123623177875) - rollout_corr/kl:np.float64(0.00038778805755357304) - rollout_corr/k3_kl:np.float64(0.0005165411663199393) - rollout_corr/rollout_ppl:np.float64(1.3510792162269354) - rollout_corr/rollout_log_ppl:np.float64(0.277003448762116) - rollout_corr/log_ppl_diff:np.float64(0.0003877874696627259) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011570610222406685) - rollout_corr/log_ppl_diff_max:np.float64(0.00403740257024765) - rollout_corr/log_ppl_diff_min:np.float64(-0.005538493394851685) - rollout_corr/ppl_ratio:np.float64(1.000388921238482) - rollout_corr/chi2_token:np.float64(0.0012935972772538662) - rollout_corr/chi2_seq:np.float64(2.2381823365576565) - actor/pg_loss:np.float64(5.385885015130043e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015849828230329877) - actor/ppo_kl:np.float64(8.225317838395085e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11362059134989977) - perf/mfu/actor:np.float64(0.1031450502424441) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.59148025512695) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007041455246508121 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007041455246508121 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:895.921875 - response_length/max:4074.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:895.921875 - response_length_non_aborted/max:4074.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.657893002033234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2360860873013735) - timing_s/agent_loop/generate_sequences/max:np.float64(26.039900217205286) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.459900658301194) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.039900217205286) - timing_s/agen
(TaskRunner pid=2049544) Training Progress: 96%|█████████▌| 96/100 [1:16:52<03:16, 49.09s/it]
(TaskRunner pid=2049544) step:97 - global_seqlen/min:20006 - global_seqlen/max:63623 - global_seqlen/minmax_diff:43617 - global_seqlen/balanced_min:40703 - global_seqlen/balanced_max:40790 - global_seqlen/mean:40767.5 - actor/entropy:0.4114430248737335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3968365788459778 - training/rollout_probs_diff_mean:0.0036807965952903032 - training/rollout_probs_diff_std:0.009112636558711529 - training/rollout_actor_probs_pearson_corr:0.9994710087776184 - rollout_corr/rollout_is_mean:0.9999697208404541 - rollout_corr/rollout_is_ratio_fraction_high:3.885426394845126e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.5541705579380505e-05 - rollout_corr/rollout_is_max:6.432518005371094 - rollout_corr/rollout_is_min:0.14259245991706848 - rollout_corr/rollout_is_std:0.030860379338264465 - rollout_corr/rollout_is_eff_sample_size:0.9990483051530745 - rollout_corr/rollout_is_seq_mean:1.000016450881958 - rollout_corr/rollout_is_seq_std:0.0015233448939397931 - rollout_corr/rollout_is_seq_max:1.0063254833221436 - rollout_corr/rollout_is_seq_min:0.994326114654541 - rollout_corr/rollout_is_seq_max_deviation:0.006325483322143555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3867830368690193) - rollout_corr/training_log_ppl:np.float64(0.30247966502793133) - rollout_corr/kl:np.float64(0.0005262127845906406) - rollout_corr/k3_kl:np.float64(0.0005436497510515892) - rollout_corr/rollout_ppl:np.float64(1.386001723818481) - rollout_corr/rollout_log_ppl:np.float64(0.30195345119864214) - rollout_corr/log_ppl_diff:np.float64(0.0005262138292891905) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011905098654096946) - rollout_corr/log_ppl_diff_max:np.float64(0.007206156849861145) - rollout_corr/log_ppl_diff_min:np.float64(-0.003447234630584717) - rollout_corr/ppl_ratio:np.float64(1.0005276133306324) - rollout_corr/chi2_token:np.float64(0.001151977339759469) - rollout_corr/chi2_seq:np.float64(0.950474871089682) - actor/pg_loss:np.float64(6.92273024469614e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019685469499108876) - actor/ppo_kl:np.float64(3.492400511362348e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16588949039578438) - perf/mfu/actor:np.float64(0.11018383492360453) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.53912734985352) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.043718818575143814 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.043718818575143814 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1005.359375 - response_length/max:4322.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1005.359375 - response_length_non_aborted/max:4322.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012983568012714386 - timing_s/agent_loop/generate_sequences/min:np.float64(1.021046968176961) - timing_s/agent_loop/generate_sequences/max:np.float64(28.757525622844696) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.163475782763271) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.757525622844696) - timing_s/age
(TaskRunner pid=2049544) Training Progress: 97%|█████████▋| 97/100 [1:17:40<02:27, 49.03s/it]
(TaskRunner pid=2049544) step:98 - global_seqlen/min:23082 - global_seqlen/max:43864 - global_seqlen/minmax_diff:20782 - global_seqlen/balanced_min:32541 - global_seqlen/balanced_max:33196 - global_seqlen/mean:32691.375 - actor/entropy:0.3132493495941162 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6835976243019104 - training/rollout_probs_diff_mean:0.0034196285996586084 - training/rollout_probs_diff_std:0.009468399919569492 - training/rollout_actor_probs_pearson_corr:0.9993211627006531 - rollout_corr/rollout_is_mean:1.0001163482666016 - rollout_corr/rollout_is_ratio_fraction_high:1.5585144865326583e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5585144865326583e-05 - rollout_corr/rollout_is_max:5.03674840927124 - rollout_corr/rollout_is_min:0.0648747980594635 - rollout_corr/rollout_is_std:0.03038937970995903 - rollout_corr/rollout_is_eff_sample_size:0.9990775757045947 - rollout_corr/rollout_is_seq_mean:1.0001397132873535 - rollout_corr/rollout_is_seq_std:0.001354765729047358 - rollout_corr/rollout_is_seq_max:1.0069159269332886 - rollout_corr/rollout_is_seq_min:0.9959137439727783 - rollout_corr/rollout_is_seq_max_deviation:0.006915926933288574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2990983063355088) - rollout_corr/training_log_ppl:np.float64(0.24720221558527555) - rollout_corr/kl:np.float64(0.0005138034307572426) - rollout_corr/k3_kl:np.float64(0.0005089092589116717) - rollout_corr/rollout_ppl:np.float64(1.298400120344013) - rollout_corr/rollout_log_ppl:np.float64(0.24668841203674674) - rollout_corr/log_ppl_diff:np.float64(0.0005138035485288128) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011324260121909901) - rollout_corr/log_ppl_diff_max:np.float64(0.00813785195350647) - rollout_corr/log_ppl_diff_min:np.float64(-0.005736410617828369) - rollout_corr/ppl_ratio:np.float64(1.0005151100922376) - rollout_corr/chi2_token:np.float64(0.0009681761730462313) - rollout_corr/chi2_seq:np.float64(4.5980356687214226) - actor/pg_loss:np.float64(-9.167857933789492e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002514609163881687) - actor/ppo_kl:np.float64(0.00017433107315412144) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1871425360441208) - perf/mfu/actor:np.float64(0.09212177950831188) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.56390380859375) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003707965661305934 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0003707965661305934 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:751.91796875 - response_length/max:4148.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:751.91796875 - response_length_non_aborted/max:4148.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010955892503261566 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1268691644072533) - timing_s/agent_loop/generate_sequences/max:np.float64(24.762351796031) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.324899325562001) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.762351796031) - timing_s/ag
(TaskRunner pid=2049544) Training Progress: 98%|█████████▊| 98/100 [1:18:24<01:34, 47.50s/it]
(TaskRunner pid=2049544) step:99 - global_seqlen/min:21119 - global_seqlen/max:73608 - global_seqlen/minmax_diff:52489 - global_seqlen/balanced_min:37066 - global_seqlen/balanced_max:37818 - global_seqlen/mean:37233.375 - actor/entropy:0.36280471086502075 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29852432012557983 - training/rollout_probs_diff_mean:0.0034003094770014286 - training/rollout_probs_diff_std:0.009052536450326443 - training/rollout_actor_probs_pearson_corr:0.9994275569915771 - rollout_corr/rollout_is_mean:0.9999655485153198 - rollout_corr/rollout_is_ratio_fraction_high:4.338112375990022e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.7352449503960088e-05 - rollout_corr/rollout_is_max:2.0474398136138916 - rollout_corr/rollout_is_min:0.40364018082618713 - rollout_corr/rollout_is_std:0.029825152829289436 - rollout_corr/rollout_is_eff_sample_size:0.9991112508590329 - rollout_corr/rollout_is_seq_mean:0.9999865293502808 - rollout_corr/rollout_is_seq_std:0.0013986814301460981 - rollout_corr/rollout_is_seq_max:1.0060322284698486 - rollout_corr/rollout_is_seq_min:0.9942169785499573 - rollout_corr/rollout_is_seq_max_deviation:0.006032228469848633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2896207650192082) - rollout_corr/training_log_ppl:np.float64(0.2305254971361137) - rollout_corr/kl:np.float64(0.0005066159342224097) - rollout_corr/k3_kl:np.float64(0.0004654821964606981) - rollout_corr/rollout_ppl:np.float64(1.2889360883273184) - rollout_corr/rollout_log_ppl:np.float64(0.23001887879217975) - rollout_corr/log_ppl_diff:np.float64(0.0005066183439339511) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010047785108326934) - rollout_corr/log_ppl_diff_max:np.float64(0.0074303895235061646) - rollout_corr/log_ppl_diff_min:np.float64(-0.004489123821258545) - rollout_corr/ppl_ratio:np.float64(1.0005076131783426) - rollout_corr/chi2_token:np.float64(0.000825623283162713) - rollout_corr/chi2_seq:np.float64(0.5253964632283896) - actor/pg_loss:np.float64(7.684435695409775e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010054797257907921) - actor/ppo_kl:np.float64(4.189955548028479e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08311776397749782) - perf/mfu/actor:np.float64(0.10069730946959708) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.47393417358398) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.86328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.86328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017708174884319305 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017708174884319305 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:900.44921875 - response_length/max:6057.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:900.44921875 - response_length_non_aborted/max:6057.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011433474719524384 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8707099817693233) - timing_s/agent_loop/generate_sequences/max:np.float64(35.631088424474) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.754026362330478) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(35.631088424474) - t
(TaskRunner pid=2049544) Training Progress: 99%|█████████▉| 99/100 [1:19:20<00:49, 49.85s/it]
(TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100}
(TaskRunner pid=2049544) validation generation end
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) [prompt] system
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format:
(TaskRunner pid=2049544) <reasoning>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) ...
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output:
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) A
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) user
(TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) A: 50 pC
(TaskRunner pid=2049544) B: 56 pC
(TaskRunner pid=2049544) C: 60 pC
(TaskRunner pid=2049544) D: 64 pC
(TaskRunner pid=2049544) Please reason step by step.
(TaskRunner pid=2049544) assistant
(TaskRunner pid=2049544) <think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </think>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) [response] <reasoning>
(TaskRunner pid=2049544) We are given:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $
(TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $
(TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) We use the formula for the electric field due to a point charge:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) E = k \frac{q}{r^2}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Solving for $ q $:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{E r^2}{k}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Substitute the values:
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C):
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC}
(TaskRunner pid=2049544) $$
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) The closest answer is **B: 56 pC**.
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) </reasoning>
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) <answer>
(TaskRunner pid=2049544) B
(TaskRunner pid=2049544) </answer>
(TaskRunner pid=2049544) [ground_truth] B
(TaskRunner pid=2049544) [score] 1.0
(TaskRunner pid=2049544) [acc] 1.0
(TaskRunner pid=2049544) [pred] B
(TaskRunner pid=2049544) [incorrect_format] 1
(TaskRunner pid=2049544) [feedback]
(TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100
(WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor
(TaskRunner pid=2049544) step:100 - global_seqlen/min:17010 - global_seqlen/max:43433 - global_seqlen/minmax_diff:26423 - global_seqlen/balanced_min:31299 - global_seqlen/balanced_max:31491 - global_seqlen/mean:31419.0 - actor/entropy:0.37953582406044006 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.630595862865448 - training/rollout_probs_diff_mean:0.0036211644764989614 - training/rollout_probs_diff_std:0.00978316180408001 - training/rollout_actor_probs_pearson_corr:0.999374508857727 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:1.0963950444420334e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.644592521188315e-05 - rollout_corr/rollout_is_max:2.3022942543029785 - rollout_corr/rollout_is_min:0.3131349980831146 - rollout_corr/rollout_is_std:0.031683772802352905 - rollout_corr/rollout_is_eff_sample_size:0.998997264270168 - rollout_corr/rollout_is_seq_mean:1.0000271797180176 - rollout_corr/rollout_is_seq_std:0.0016033316496759653 - rollout_corr/rollout_is_seq_max:1.0072500705718994 - rollout_corr/rollout_is_seq_min:0.9959959983825684 - rollout_corr/rollout_is_seq_max_deviation:0.007250070571899414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3458213401027024) - rollout_corr/training_log_ppl:np.float64(0.26966726542741526) - rollout_corr/kl:np.float64(0.00037315004849824973) - rollout_corr/k3_kl:np.float64(0.0005444729254122649) - rollout_corr/rollout_ppl:np.float64(1.3452776027843356) - rollout_corr/rollout_log_ppl:np.float64(0.2692941162531497) - rollout_corr/log_ppl_diff:np.float64(0.0003731491742655635) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012809229083359241) - rollout_corr/log_ppl_diff_max:np.float64(0.0049625784158706665) - rollout_corr/log_ppl_diff_min:np.float64(-0.0106421560049057) - rollout_corr/ppl_ratio:np.float64(1.0003747479058802) - rollout_corr/chi2_token:np.float64(0.0014326744712889194) - rollout_corr/chi2_seq:np.float64(1.9170411999803036) - actor/pg_loss:np.float64(-7.347844075411558e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001629320678375734) - actor/ppo_kl:np.float64(-0.00012778303023264215) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12031856318935752) - perf/mfu/actor:np.float64(0.0881824181802986) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.57986450195312) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/reward/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08949529688241312) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8910875) - val-aux/sciknoweval/reward/best@1
(TaskRunner pid=2049544) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': "
(TaskRunner pid=2049544) "np.float64(0.6828125), 'val-aux/sciknoweval/reward/std@16': "
(TaskRunner pid=2049544) "np.float64(0.17979486326514935), 'val-aux/sciknoweval/reward/best@2/mean': "
(TaskRunner pid=2049544) "np.float64(0.7532), 'val-aux/sciknoweval/reward/best@2/std': "
(TaskRunner pid=2049544) "np.float64(0.1476854115376502), 'val-aux/sciknoweval/reward/worst@2/mean': "
(TaskRunner pid=2049544) "np.float64(0.6133875), 'val-aux/sciknoweval/reward/worst@2/std': "
(TaskRunner pid=2049544) "np.float64(0.15413287282151064), 'val-aux/sciknoweval/reward/maj@2/mean': "
(TaskRunner pid=2049544) "np.float64(0.6832499999999999), 'val-aux/sciknoweval/reward/maj@2/std': "
(TaskRunner pid=2049544) "np.float64(0.17928347494256552), 'val-aux/sciknoweval/reward/best@4/mean': "
(TaskRunner pid=2049544) "np.float64(0.807675), 'val-aux/sciknoweval/reward/best@4/std': "
(TaskRunner pid=2049544) "np.float64(0.11309093981217563), 'val-aux/sciknoweval/reward/worst@4/mean': "
(TaskRunner pid=2049544) "np.float64(0.5515125000000001), 'val-aux/sciknoweval/reward/worst@4/std': "
(TaskRunner pid=2049544) "np.float64(0.12241577016075818), 'val-aux/sciknoweval/reward/maj@4/mean': "
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) Training Progress: 100%|██████████| 100/100 [1:22:07<00:00, 85.21s/it]
(TaskRunner pid=2049544) "np.float64(0.7018125), 'val-aux/sciknoweval/reward/maj@4/std': "
(TaskRunner pid=2049544) "np.float64(0.13581544160753714), 'val-aux/sciknoweval/reward/best@8/mean': "
(TaskRunner pid=2049544) "np.float64(0.8526375000000002), 'val-aux/sciknoweval/reward/best@8/std': "
(TaskRunner pid=2049544) "np.float64(0.0888708326383256), 'val-aux/sciknoweval/reward/worst@8/mean': "
(TaskRunner pid=2049544) "np.float64(0.4995375), 'val-aux/sciknoweval/reward/worst@8/std': "
(TaskRunner pid=2049544) "np.float64(0.08946281937139845), 'val-aux/sciknoweval/reward/maj@8/mean': "
(TaskRunner pid=2049544) "np.float64(0.710175), 'val-aux/sciknoweval/reward/maj@8/std': "
(TaskRunner pid=2049544) "np.float64(0.08949529688241312), 'val-aux/sciknoweval/reward/best@16/mean': "
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) "np.float64(0.8910875), 'val-aux/sciknoweval/reward/best@16/std': "
(TaskRunner pid=2049544) 'np.float64(0.061136881006381635), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.46395), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.054003469751280585), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7157250000000001), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.05584489583302406), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/mean@16': np.float64(0.6828125), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/std@16': np.float64(0.17979486326514935), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.7532), "
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.1476854115376502), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.6133875), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.15413287282151064), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.6832499999999999), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.17928347494256552), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.807675), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.11309093981217563), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.5515125000000001), "
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.12241577016075818), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.7018125), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.13581544160753714), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8526375000000002), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.0888708326383256), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.4995375), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.08946281937139845), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.710175), "
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.08949529688241312), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.8910875), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.061136881006381635), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.46395), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.054003469751280585), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7157250000000001), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.05584489583302406), "
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) "'val-core/sciknoweval/acc/mean@16': np.float64(0.6828125), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/std@16': np.float64(0.17979486326514935), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.7532), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.1476854115376502), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.6133875), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.15413287282151064), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.6832499999999999), "
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.17928347494256552), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.807675), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.11309093981217563), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.5515125000000001), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.12241577016075818), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.7018125), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.13581544160753714), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8526375000000002), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.0888708326383256), "
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.4995375), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.08946281937139845), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.710175), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.08949529688241312), "
(TaskRunner pid=2049544) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.8910875), "
(TaskRunner pid=2049544) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.061136881006381635), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.46395), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.054003469751280585), "
(TaskRunner pid=2049544) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7157250000000001), "
(TaskRunner pid=2049544) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.05584489583302406), "
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.9859375), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/std@16': "
(TaskRunner pid=2049544) 'np.float64(0.03383512807052093), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(0.99745), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@2/std': "
(TaskRunner pid=2049544) 'np.float64(0.014287439995541685), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': "
(TaskRunner pid=2049544) 'np.float64(0.9741250000000001), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@2/std': "
(TaskRunner pid=2049544) 'np.float64(0.042452979144002284), '
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9857875), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@2/std': "
(TaskRunner pid=2049544) 'np.float64(0.03387729307844336), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@4/mean': "
(TaskRunner pid=2049544) 'np.float64(0.9998374999999999), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@4/std': "
(TaskRunner pid=2049544) 'np.float64(0.002749370152057837), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(0.9560875), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@4/std': "
(TaskRunner pid=2049544) 'np.float64(0.04764508939236297), '
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': "
(TaskRunner pid=2049544) 'np.float64(0.9946999999999999), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@4/std': "
(TaskRunner pid=2049544) 'np.float64(0.021483511676811038), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(0.933025), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@8/std': "
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) 'np.float64(0.0436722700672832), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': "
(TaskRunner pid=2049544) 'np.float64(0.9988624999999999), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@8/std': "
(TaskRunner pid=2049544) 'np.float64(0.009237722502874987), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), "
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': "
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) 'np.float64(0.9139250000000001), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@16/std': "
(TaskRunner pid=2049544) 'np.float64(0.029568343249240757), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': "
(TaskRunner pid=2049544) 'np.float64(0.9998750000000001), '
(TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@16/std': "
(TaskRunner pid=2049544) "np.float64(0.0017543234207183663), 'val-aux/num_turns/min': np.int32(2), "
(TaskRunner pid=2049544) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': "
(TaskRunner pid=2049544) 'np.float64(2.0)}')
(TaskRunner pid=2049544) Training Progress: 100%|██████████| 100/100 [1:22:07<00:00, 49.28s/it]
(TaskRunner pid=2049544)
(TaskRunner pid=2049544) wandb: updating run metadata
(TaskRunner pid=2049544) wandb: uploading output.log; uploading wandb-summary.json
(TaskRunner pid=2049544) wandb: uploading output.log; uploading config.yaml
(TaskRunner pid=2049544) wandb: uploading history steps 98-99, summary, console lines 919-1044
(TaskRunner pid=2049544) wandb:
(TaskRunner pid=2049544) wandb: Run history:
(TaskRunner pid=2049544) wandb: actor/entropy ▁▁▂▁▂▁▁▁▂▃▃▂▂▂▃▂▃▃▃▃▄▄▃▄▅▄▆▄▇▆▅▄▆▆▆▅▇█▇▇
(TaskRunner pid=2049544) wandb: actor/grad_norm █▄▇▃█▆▃▇▆▁▄▄▃▃▃▂▁▂▁▂▄▃▂▂▂▂▄▂▃▃▂▃▂▃▂▁▃▂▂▁
(TaskRunner pid=2049544) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
(TaskRunner pid=2049544) wandb: actor/lr ▁▂▃▄▆███████████████████████████████████
(TaskRunner pid=2049544) wandb: actor/pg_clipfrac ▂▄▅▃▆▇▄▃█▃▃▄▂▂▃▃▂▂▂▂▃▃▃▂▂▁▁▁▁▂▁▁▁▂▂▂▁▁▂▁
(TaskRunner pid=2049544) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▆▁▁▄▁▃▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▁
(TaskRunner pid=2049544) wandb: actor/pg_loss ▆▂▅▅█▆▅▃▃▅▅▄▃▄▅▄▄▃▂▅▁▂▄▄▁▄▆▃▅▄▅▅▂▅▅▄▅▃▅▃
(TaskRunner pid=2049544) wandb: actor/ppo_kl ▁▂▂▃▄▅▃▁▄▃▄▂▄▁▂▂▃▃▂▂▇▄▂▃▂▂▄▄▃▁▄▃▁▂█▃▃▂▃▂
(TaskRunner pid=2049544) wandb: critic/advantages/max █████████▅█████████████▅███▁████████████
(TaskRunner pid=2049544) wandb: critic/advantages/mean ▅▇▆▅▂▄▃▅▅▅▃▂▃▆▃▇▆▂█▂▄▅▃▃▅▁▅▄▄▄▄▅▄▆▆▅▄▃▂▅
(TaskRunner pid=2049544) wandb: +204 ...
(TaskRunner pid=2049544) wandb:
(TaskRunner pid=2049544) wandb: Run summary:
(TaskRunner pid=2049544) wandb: actor/entropy 0.37954
(TaskRunner pid=2049544) wandb: actor/grad_norm 0.12032
(TaskRunner pid=2049544) wandb: actor/kl_loss 0
(TaskRunner pid=2049544) wandb: actor/lr 0.0
(TaskRunner pid=2049544) wandb: actor/pg_clipfrac 0.00016
(TaskRunner pid=2049544) wandb: actor/pg_clipfrac_lower 0
(TaskRunner pid=2049544) wandb: actor/pg_loss -7e-05
(TaskRunner pid=2049544) wandb: actor/ppo_kl -0.00013
(TaskRunner pid=2049544) wandb: critic/advantages/max 0.875
(TaskRunner pid=2049544) wandb: critic/advantages/mean -0.00182
(TaskRunner pid=2049544) wandb: +204 ...
(TaskRunner pid=2049544) wandb:
(TaskRunner pid=2049544) wandb: 🚀 View run qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/o8ivivjg
(TaskRunner pid=2049544) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root
(TaskRunner pid=2049544) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
(TaskRunner pid=2049544) wandb: Find logs at: ./wandb/run-20260702_073403-o8ivivjg/logs
main_ppo exit code: 0
[2026-07-02 08:56:17] Finished physics grpo
[2026-07-02 08:56:17] Starting physics rlsd_tr
Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8
Dataset: physics
Method: rlsd_tr
Config: rlsd
Model: Qwen/Qwen3-4B
Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet
Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet
Ray tmp: /tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B
2026-07-02 08:56:19,506 INFO scripts.py:1364 -- Did not find any active Ray processes.
Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8
Config: rlsd
Task: datasets/sciknoweval/physics
Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240
ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}}
2026-07-02 08:56:29,254 INFO worker.py:2007 -- Started a local Ray instance.
/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
warnings.warn(
(TaskRunner pid=2074004) TaskRunner hostname: mole-workspace-rw, PID: 2074004
(TaskRunner pid=2074004) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2074004) 'calculate_entropy': False,
(TaskRunner pid=2074004) 'calculate_sum_pi_squared': False,
(TaskRunner pid=2074004) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2074004) 'async_save': False,
(TaskRunner pid=2074004) 'load_contents': ['model',
(TaskRunner pid=2074004) 'optimizer',
(TaskRunner pid=2074004) 'extra'],
(TaskRunner pid=2074004) 'save_contents': ['model',
(TaskRunner pid=2074004) 'optimizer',
(TaskRunner pid=2074004) 'extra']},
(TaskRunner pid=2074004) 'clip_ratio': 0.2,
(TaskRunner pid=2074004) 'clip_ratio_c': 3.0,
(TaskRunner pid=2074004) 'clip_ratio_high': 0.28,
(TaskRunner pid=2074004) 'clip_ratio_low': 0.2,
(TaskRunner pid=2074004) 'data_loader_seed': 42,
(TaskRunner pid=2074004) 'entropy_checkpointing': False,
(TaskRunner pid=2074004) 'entropy_coeff': 0,
(TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2074004) 'freeze_vision_tower': False,
(TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2074004) 'dtype': 'bfloat16',
(TaskRunner pid=2074004) 'entropy_checkpointing': False,
(TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2074004) 'forward_only': False,
(TaskRunner pid=2074004) 'forward_prefetch': False,
(TaskRunner pid=2074004) 'fsdp_size': -1,
(TaskRunner pid=2074004) 'full_determinism': False,
(TaskRunner pid=2074004) 'model_dtype': 'fp32',
(TaskRunner pid=2074004) 'offload_policy': False,
(TaskRunner pid=2074004) 'optimizer_offload': False,
(TaskRunner pid=2074004) 'param_offload': False,
(TaskRunner pid=2074004) 'reshard_after_forward': True,
(TaskRunner pid=2074004) 'seed': 42,
(TaskRunner pid=2074004) 'strategy': 'fsdp',
(TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2074004) 'use_orig_params': False,
(TaskRunner pid=2074004) 'use_torch_compile': True,
(TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2074004) 'grad_clip': 1.0,
(TaskRunner pid=2074004) 'kl_loss_coef': 0.001,
(TaskRunner pid=2074004) 'kl_loss_type': 'low_var_kl',
(TaskRunner pid=2074004) 'loss_agg_mode': 'token-mean',
(TaskRunner pid=2074004) 'loss_scale_factor': None,
(TaskRunner pid=2074004) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2074004) 'betas': [0.9, 0.999],
(TaskRunner pid=2074004) 'clip_grad': 1.0,
(TaskRunner pid=2074004) 'lr': 1e-06,
(TaskRunner pid=2074004) 'lr_scheduler_type': 'constant',
(TaskRunner pid=2074004) 'lr_warmup_steps': 10,
(TaskRunner pid=2074004) 'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2074004) 'min_lr_ratio': 0.0,
(TaskRunner pid=2074004) 'num_cycles': 0.5,
(TaskRunner pid=2074004) 'optimizer': 'AdamW',
(TaskRunner pid=2074004) 'optimizer_impl': 'torch.optim',
(TaskRunner pid=2074004) 'override_optimizer_config': None,
(TaskRunner pid=2074004) 'total_training_steps': -1,
(TaskRunner pid=2074004) 'warmup_style': None,
(TaskRunner pid=2074004) 'weight_decay': 0.01},
(TaskRunner pid=2074004) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig',
(TaskRunner pid=2074004) 'clip_cov_lb': 1.0,
(TaskRunner pid=2074004) 'clip_cov_ratio': 0.0002,
(TaskRunner pid=2074004) 'clip_cov_ub': 5.0,
(TaskRunner pid=2074004) 'kl_cov_ratio': 0.0002,
(TaskRunner pid=2074004) 'loss_mode': 'rlsd',
(TaskRunner pid=2074004) 'ppo_kl_coef': 0.1},
(TaskRunner pid=2074004) 'ppo_epochs': 1,
(TaskRunner pid=2074004) 'ppo_max_token_len_per_gpu': 10240,
(TaskRunner pid=2074004) 'ppo_micro_batch_size': None,
(TaskRunner pid=2074004) 'ppo_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2074004) 'ppo_mini_batch_size': 8,
(TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2074004) 'all_ranks': False,
(TaskRunner pid=2074004) 'enable': False,
(TaskRunner pid=2074004) 'ranks': [],
(TaskRunner pid=2074004) 'save_path': 'outputs/profile',
(TaskRunner pid=2074004) 'tool': None,
(TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2074004) 'analysis': True,
(TaskRunner pid=2074004) 'contents': [],
(TaskRunner pid=2074004) 'discrete': False,
(TaskRunner pid=2074004) 'level': 'level0'},
(TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2074004) 'discrete': False},
(TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2074004) 'step_end': None,
(TaskRunner pid=2074004) 'step_start': 0},
(TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2074004) 'stack_depth': 32,
(TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2074004) 'rollout_n': 8,
(TaskRunner pid=2074004) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2074004) 'mode': 'disabled',
(TaskRunner pid=2074004) 'record_file': None,
(TaskRunner pid=2074004) 'replay_file': None},
(TaskRunner pid=2074004) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig',
(TaskRunner pid=2074004) 'alpha': 0.5,
(TaskRunner pid=2074004) 'distillation_add_tail': True,
(TaskRunner pid=2074004) 'distillation_topk': 100,
(TaskRunner pid=2074004) 'dont_reprompt_on_self_success': True,
(TaskRunner pid=2074004) 'environment_feedback_only_without_solution': True,
(TaskRunner pid=2074004) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig',
(TaskRunner pid=2074004) 'enable': False,
(TaskRunner pid=2074004) 'loss_weight': 0.0,
(TaskRunner pid=2074004) 'mask': 'all'},
(TaskRunner pid=2074004) 'feedback_template': '\n'
(TaskRunner pid=2074004) 'The '
(TaskRunner pid=2074004) 'following '
(TaskRunner pid=2074004) 'is '
(TaskRunner pid=2074004) 'feedback '
(TaskRunner pid=2074004) 'from '
(TaskRunner pid=2074004) 'your '
(TaskRunner pid=2074004) 'unsuccessful '
(TaskRunner pid=2074004) 'earlier '
(TaskRunner pid=2074004) 'attempt:\n'
(TaskRunner pid=2074004) '\n'
(TaskRunner pid=2074004) '{feedback_raw}',
(TaskRunner pid=2074004) 'full_logit_distillation': True,
(TaskRunner pid=2074004) 'include_environment_feedback': True,
(TaskRunner pid=2074004) 'is_clip': 2,
(TaskRunner pid=2074004) 'max_reprompt_len': 10240,
(TaskRunner pid=2074004) 'remove_thinking_from_demonstration': False,
(TaskRunner pid=2074004) 'reprompt_template': '{prompt}{solution}{feedback}\n'
(TaskRunner pid=2074004) '\n'
(TaskRunner pid=2074004) 'Correctly '
(TaskRunner pid=2074004) 'solve '
(TaskRunner pid=2074004) 'the '
(TaskRunner pid=2074004) 'original '
(TaskRunner pid=2074004) 'question.',
(TaskRunner pid=2074004) 'reprompt_truncation': 'right',
(TaskRunner pid=2074004) 'solution_template': '\n'
(TaskRunner pid=2074004) 'Correct '
(TaskRunner pid=2074004) 'solution:\n'
(TaskRunner pid=2074004) '\n'
(TaskRunner pid=2074004) '{successful_previous_attempt}',
(TaskRunner pid=2074004) 'srpo_dynamic_weighting': False,
(TaskRunner pid=2074004) 'srpo_dynamic_weighting_temperature': 1.0,
(TaskRunner pid=2074004) 'success_reward_threshold': 0.5,
(TaskRunner pid=2074004) 'teacher_regularization': 'trust-region',
(TaskRunner pid=2074004) 'teacher_update_rate': 0.1,
(TaskRunner pid=2074004) 'token_reweight_decay_steps': 0,
(TaskRunner pid=2074004) 'token_reweight_eps_w': 0.2,
(TaskRunner pid=2074004) 'token_reweight_lambda': 0.5},
(TaskRunner pid=2074004) 'shuffle': False,
(TaskRunner pid=2074004) 'strategy': 'fsdp',
(TaskRunner pid=2074004) 'sum_pi_squared_checkpointing': False,
(TaskRunner pid=2074004) 'tau_neg': 1.05,
(TaskRunner pid=2074004) 'tau_pos': 1.0,
(TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2074004) 'use_dynamic_bsz': False,
(TaskRunner pid=2074004) 'use_fused_kernels': False,
(TaskRunner pid=2074004) 'use_kl_loss': False,
(TaskRunner pid=2074004) 'use_prefix_grouper': False,
(TaskRunner pid=2074004) 'use_remove_padding': True,
(TaskRunner pid=2074004) 'use_torch_compile': True},
(TaskRunner pid=2074004) 'hybrid_engine': True,
(TaskRunner pid=2074004) 'model': {'_target_': 'verl.workers.config.HFModelConfig',
(TaskRunner pid=2074004) 'custom_chat_template': None,
(TaskRunner pid=2074004) 'enable_activation_offload': False,
(TaskRunner pid=2074004) 'enable_gradient_checkpointing': True,
(TaskRunner pid=2074004) 'exclude_modules': None,
(TaskRunner pid=2074004) 'external_lib': None,
(TaskRunner pid=2074004) 'fused_kernel_options': {'impl_backend': 'torch'},
(TaskRunner pid=2074004) 'hf_config_path': None,
(TaskRunner pid=2074004) 'lora_adapter_path': None,
(TaskRunner pid=2074004) 'lora_alpha': 16,
(TaskRunner pid=2074004) 'lora_rank': 0,
(TaskRunner pid=2074004) 'override_config': {},
(TaskRunner pid=2074004) 'path': 'Qwen/Qwen3-4B',
(TaskRunner pid=2074004) 'target_modules': 'all-linear',
(TaskRunner pid=2074004) 'tiled_mlp': {'enabled': False,
(TaskRunner pid=2074004) 'num_shards': 4},
(TaskRunner pid=2074004) 'tokenizer_path': None,
(TaskRunner pid=2074004) 'trust_remote_code': True,
(TaskRunner pid=2074004) 'use_fused_kernels': False,
(TaskRunner pid=2074004) 'use_liger': False,
(TaskRunner pid=2074004) 'use_remove_padding': True,
(TaskRunner pid=2074004) 'use_shm': False},
(TaskRunner pid=2074004) 'nccl_timeout': 600,
(TaskRunner pid=2074004) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig',
(TaskRunner pid=2074004) 'entropy_checkpointing': False,
(TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2074004) 'dtype': 'bfloat16',
(TaskRunner pid=2074004) 'entropy_checkpointing': False,
(TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2074004) 'forward_only': True,
(TaskRunner pid=2074004) 'forward_prefetch': False,
(TaskRunner pid=2074004) 'fsdp_size': -1,
(TaskRunner pid=2074004) 'full_determinism': False,
(TaskRunner pid=2074004) 'model_dtype': 'fp32',
(TaskRunner pid=2074004) 'offload_policy': False,
(TaskRunner pid=2074004) 'optimizer_offload': False,
(TaskRunner pid=2074004) 'param_offload': False,
(TaskRunner pid=2074004) 'reshard_after_forward': True,
(TaskRunner pid=2074004) 'seed': 42,
(TaskRunner pid=2074004) 'strategy': 'fsdp',
(TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2074004) 'use_orig_params': False,
(TaskRunner pid=2074004) 'use_torch_compile': True,
(TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2074004) 'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2074004) 'log_prob_micro_batch_size': None,
(TaskRunner pid=2074004) 'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2074004) 'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2074004) 'all_ranks': False,
(TaskRunner pid=2074004) 'enable': False,
(TaskRunner pid=2074004) 'ranks': [],
(TaskRunner pid=2074004) 'save_path': 'outputs/profile',
(TaskRunner pid=2074004) 'tool': None,
(TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2074004) 'analysis': True,
(TaskRunner pid=2074004) 'contents': [],
(TaskRunner pid=2074004) 'discrete': False,
(TaskRunner pid=2074004) 'level': 'level0'},
(TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2074004) 'discrete': False},
(TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2074004) 'step_end': None,
(TaskRunner pid=2074004) 'step_start': 0},
(TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2074004) 'stack_depth': 32,
(TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2074004) 'rollout_n': 8,
(TaskRunner pid=2074004) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig',
(TaskRunner pid=2074004) 'mode': 'disabled',
(TaskRunner pid=2074004) 'record_file': None,
(TaskRunner pid=2074004) 'replay_file': None},
(TaskRunner pid=2074004) 'strategy': 'fsdp',
(TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2074004) 'use_torch_compile': True},
(TaskRunner pid=2074004) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2074004) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig',
(TaskRunner pid=2074004) 'agent_loop_config_path': None,
(TaskRunner pid=2074004) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig',
(TaskRunner pid=2074004) 'name': None,
(TaskRunner pid=2074004) 'path': None},
(TaskRunner pid=2074004) 'default_agent_loop': 'single_turn_agent',
(TaskRunner pid=2074004) 'num_workers': 8},
(TaskRunner pid=2074004) 'calculate_log_probs': True,
(TaskRunner pid=2074004) 'cudagraph_capture_sizes': None,
(TaskRunner pid=2074004) 'data_parallel_size': 1,
(TaskRunner pid=2074004) 'disable_log_stats': True,
(TaskRunner pid=2074004) 'do_sample': True,
(TaskRunner pid=2074004) 'dtype': 'bfloat16',
(TaskRunner pid=2074004) 'enable_chunked_prefill': True,
(TaskRunner pid=2074004) 'enable_prefix_caching': True,
(TaskRunner pid=2074004) 'enable_rollout_routing_replay': False,
(TaskRunner pid=2074004) 'enforce_eager': False,
(TaskRunner pid=2074004) 'engine_kwargs': {'sglang': {}, 'vllm': {}},
(TaskRunner pid=2074004) 'expert_parallel_size': 1,
(TaskRunner pid=2074004) 'free_cache_engine': True,
(TaskRunner pid=2074004) 'gpu_memory_utilization': 0.8,
(TaskRunner pid=2074004) 'ignore_eos': False,
(TaskRunner pid=2074004) 'layered_summon': False,
(TaskRunner pid=2074004) 'load_format': 'dummy',
(TaskRunner pid=2074004) 'log_prob_max_token_len_per_gpu': 10240,
(TaskRunner pid=2074004) 'log_prob_micro_batch_size': None,
(TaskRunner pid=2074004) 'log_prob_micro_batch_size_per_gpu': 1,
(TaskRunner pid=2074004) 'log_prob_use_dynamic_bsz': False,
(TaskRunner pid=2074004) 'logprobs_mode': 'processed_logprobs',
(TaskRunner pid=2074004) 'max_model_len': 10240,
(TaskRunner pid=2074004) 'max_num_batched_tokens': 10240,
(TaskRunner pid=2074004) 'max_num_seqs': 1024,
(TaskRunner pid=2074004) 'mode': 'async',
(TaskRunner pid=2074004) 'multi_stage_wake_up': False,
(TaskRunner pid=2074004) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig',
(TaskRunner pid=2074004) 'enable': False,
(TaskRunner pid=2074004) 'format': 'hermes',
(TaskRunner pid=2074004) 'interaction_config_path': None,
(TaskRunner pid=2074004) 'max_assistant_turns': None,
(TaskRunner pid=2074004) 'max_parallel_calls': 1,
(TaskRunner pid=2074004) 'max_tool_response_length': 256,
(TaskRunner pid=2074004) 'max_user_turns': None,
(TaskRunner pid=2074004) 'num_repeat_rollouts': None,
(TaskRunner pid=2074004) 'tokenization_sanity_check_mode': 'strict',
(TaskRunner pid=2074004) 'tool_config_path': None,
(TaskRunner pid=2074004) 'tool_response_truncate_side': 'middle',
(TaskRunner pid=2074004) 'use_inference_chat_template': False},
(TaskRunner pid=2074004) 'n': 8,
(TaskRunner pid=2074004) 'name': 'vllm',
(TaskRunner pid=2074004) 'over_sample_rate': 0,
(TaskRunner pid=2074004) 'pipeline_model_parallel_size': 1,
(TaskRunner pid=2074004) 'profiler':
(TaskRunner pid=2074004) {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2074004) 'all_ranks': False,
(TaskRunner pid=2074004) 'enable': False,
(TaskRunner pid=2074004) 'ranks': [],
(TaskRunner pid=2074004) 'save_path': 'outputs/profile',
(TaskRunner pid=2074004) 'tool': None,
(TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2074004) 'analysis': True,
(TaskRunner pid=2074004) 'contents': [],
(TaskRunner pid=2074004) 'discrete': False,
(TaskRunner pid=2074004) 'level': 'level0'},
(TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2074004) 'discrete': False},
(TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2074004) 'step_end': None,
(TaskRunner pid=2074004) 'step_start': 0},
(TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2074004) 'stack_depth': 32,
(TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2074004) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig',
(TaskRunner pid=2074004) 'enable': False,
(TaskRunner pid=2074004) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml',
(TaskRunner pid=2074004) 'port': 9090,
(TaskRunner pid=2074004) 'served_model_name': 'Qwen/Qwen3-4B'},
(TaskRunner pid=2074004) 'prompt_length': 2048,
(TaskRunner pid=2074004) 'quantization': None,
(TaskRunner pid=2074004) 'quantization_config_file': None,
(TaskRunner pid=2074004) 'response_length': 8192,
(TaskRunner pid=2074004) 'scheduling_policy': 'fcfs',
(TaskRunner pid=2074004) 'skip_dump_dir': '/tmp/rollout_dump',
(TaskRunner pid=2074004) 'skip_rollout': False,
(TaskRunner pid=2074004) 'skip_tokenizer_init': True,
(TaskRunner pid=2074004) 'temperature': 1.0,
(TaskRunner pid=2074004) 'tensor_model_parallel_size': 2,
(TaskRunner pid=2074004) 'top_k': -1,
(TaskRunner pid=2074004) 'top_p': 1.0,
(TaskRunner pid=2074004) 'trace': {'_target_': 'verl.workers.config.TraceConfig',
(TaskRunner pid=2074004) 'backend': None,
(TaskRunner pid=2074004) 'max_samples_per_step_per_worker': None,
(TaskRunner pid=2074004) 'token2text': False},
(TaskRunner pid=2074004) 'update_weights_bucket_megabytes': 512,
(TaskRunner pid=2074004) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig',
(TaskRunner pid=2074004) 'do_sample': True,
(TaskRunner pid=2074004) 'n': 16,
(TaskRunner pid=2074004) 'temperature': 0.6,
(TaskRunner pid=2074004) 'top_k': -1,
(TaskRunner pid=2074004) 'top_p': 0.95}}},
(TaskRunner pid=2074004) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig',
(TaskRunner pid=2074004) 'adv_estimator': 'grpo',
(TaskRunner pid=2074004) 'gamma': 1.0,
(TaskRunner pid=2074004) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig',
(TaskRunner pid=2074004) 'horizon': 10000,
(TaskRunner pid=2074004) 'kl_coef': 0.001,
(TaskRunner pid=2074004) 'target_kl': 0.1,
(TaskRunner pid=2074004) 'type': 'fixed'},
(TaskRunner pid=2074004) 'kl_penalty': 'kl',
(TaskRunner pid=2074004) 'lam': 1.0,
(TaskRunner pid=2074004) 'norm_adv_by_std_in_grpo': False,
(TaskRunner pid=2074004) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0},
(TaskRunner pid=2074004) 'rollout_correction': {'bypass_mode': False,
(TaskRunner pid=2074004) 'loss_type': 'ppo_clip',
(TaskRunner pid=2074004) 'rollout_is': 'token',
(TaskRunner pid=2074004) 'rollout_is_batch_normalize': False,
(TaskRunner pid=2074004) 'rollout_is_threshold': 2.0,
(TaskRunner pid=2074004) 'rollout_rs': None,
(TaskRunner pid=2074004) 'rollout_rs_threshold': None},
(TaskRunner pid=2074004) 'use_kl_in_reward': False,
(TaskRunner pid=2074004) 'use_pf_ppo': False},
(TaskRunner pid=2074004) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig',
(TaskRunner pid=2074004) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig',
(TaskRunner pid=2074004) 'async_save': False,
(TaskRunner pid=2074004) 'load_contents': ['model', 'optimizer', 'extra'],
(TaskRunner pid=2074004) 'save_contents': ['model', 'optimizer', 'extra']},
(TaskRunner pid=2074004) 'cliprange_value': 0.5,
(TaskRunner pid=2074004) 'data_loader_seed': 42,
(TaskRunner pid=2074004) 'enable': None,
(TaskRunner pid=2074004) 'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2074004) 'forward_micro_batch_size': None,
(TaskRunner pid=2074004) 'forward_micro_batch_size_per_gpu': None,
(TaskRunner pid=2074004) 'grad_clip': 1.0,
(TaskRunner pid=2074004) 'loss_agg_mode': 'token-mean',
(TaskRunner pid=2074004) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg',
(TaskRunner pid=2074004) 'enable_activation_offload': False,
(TaskRunner pid=2074004) 'enable_gradient_checkpointing': True,
(TaskRunner pid=2074004) 'external_lib': None,
(TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2074004) 'dtype': 'bfloat16',
(TaskRunner pid=2074004) 'entropy_checkpointing': False,
(TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False,
(TaskRunner pid=2074004) 'forward_only': False,
(TaskRunner pid=2074004) 'forward_prefetch': False,
(TaskRunner pid=2074004) 'fsdp_size': -1,
(TaskRunner pid=2074004) 'full_determinism': False,
(TaskRunner pid=2074004) 'model_dtype': 'fp32',
(TaskRunner pid=2074004) 'offload_policy': False,
(TaskRunner pid=2074004) 'optimizer_offload': False,
(TaskRunner pid=2074004) 'param_offload': False,
(TaskRunner pid=2074004) 'reshard_after_forward': True,
(TaskRunner pid=2074004) 'seed': 42,
(TaskRunner pid=2074004) 'strategy': 'fsdp',
(TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2074004) 'use_orig_params': False,
(TaskRunner pid=2074004) 'use_torch_compile': True,
(TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2074004) 'lora_alpha': 16,
(TaskRunner pid=2074004) 'lora_rank': 0,
(TaskRunner pid=2074004) 'override_config': {},
(TaskRunner pid=2074004) 'path': 'Qwen/Qwen3-8B',
(TaskRunner pid=2074004) 'target_modules': 'all-linear',
(TaskRunner pid=2074004) 'tiled_mlp': {'enabled': False, 'num_shards': 4},
(TaskRunner pid=2074004) 'tokenizer_path': 'Qwen/Qwen3-4B',
(TaskRunner pid=2074004) 'trust_remote_code': True,
(TaskRunner pid=2074004) 'use_remove_padding': False,
(TaskRunner pid=2074004) 'use_shm': False},
(TaskRunner pid=2074004) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig',
(TaskRunner pid=2074004) 'betas': [0.9, 0.999],
(TaskRunner pid=2074004) 'clip_grad': 1.0,
(TaskRunner pid=2074004) 'lr': 1e-05,
(TaskRunner pid=2074004) 'lr_scheduler_type': 'constant',
(TaskRunner pid=2074004) 'lr_warmup_steps': -1,
(TaskRunner pid=2074004) 'lr_warmup_steps_ratio': 0.0,
(TaskRunner pid=2074004) 'min_lr_ratio': 0.0,
(TaskRunner pid=2074004) 'num_cycles': 0.5,
(TaskRunner pid=2074004) 'optimizer': 'AdamW',
(TaskRunner pid=2074004) 'optimizer_impl': 'torch.optim',
(TaskRunner pid=2074004) 'override_optimizer_config': None,
(TaskRunner pid=2074004) 'total_training_steps': -1,
(TaskRunner pid=2074004) 'warmup_style': None,
(TaskRunner pid=2074004) 'weight_decay': 0.01},
(TaskRunner pid=2074004) 'ppo_epochs': 1,
(TaskRunner pid=2074004) 'ppo_max_token_len_per_gpu': 32768,
(TaskRunner pid=2074004) 'ppo_micro_batch_size': None,
(TaskRunner pid=2074004) 'ppo_micro_batch_size_per_gpu': None,
(TaskRunner pid=2074004) 'ppo_mini_batch_size': 8,
(TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2074004) 'all_ranks': False,
(TaskRunner pid=2074004) 'enable': False,
(TaskRunner pid=2074004) 'ranks': [],
(TaskRunner pid=2074004) 'save_path': 'outputs/profile',
(TaskRunner pid=2074004) 'tool': None,
(TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2074004) 'analysis': True,
(TaskRunner pid=2074004) 'contents': [],
(TaskRunner pid=2074004) 'discrete': False,
(TaskRunner pid=2074004) 'level': 'level0'},
(TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2074004) 'discrete': False},
(TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2074004) 'step_end': None,
(TaskRunner pid=2074004) 'step_start': 0},
(TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2074004) 'stack_depth': 32,
(TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2074004) 'rollout_n': 8,
(TaskRunner pid=2074004) 'shuffle': False,
(TaskRunner pid=2074004) 'strategy': 'fsdp',
(TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2074004) 'use_dynamic_bsz': False},
(TaskRunner pid=2074004) 'custom_reward_function': {'name': 'compute_score',
(TaskRunner pid=2074004) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'},
(TaskRunner pid=2074004) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False},
(TaskRunner pid=2074004) 'custom_cls': {'name': None, 'path': None},
(TaskRunner pid=2074004) 'datagen': {'name': None, 'path': None},
(TaskRunner pid=2074004) 'dataloader_num_workers': 8,
(TaskRunner pid=2074004) 'filter_overlong_prompts': True,
(TaskRunner pid=2074004) 'filter_overlong_prompts_workers': 1,
(TaskRunner pid=2074004) 'image_key': 'images',
(TaskRunner pid=2074004) 'image_patch_size': 14,
(TaskRunner pid=2074004) 'max_prompt_length': 2048,
(TaskRunner pid=2074004) 'max_response_length': 8192,
(TaskRunner pid=2074004) 'prompt_key': 'prompt',
(TaskRunner pid=2074004) 'return_full_prompt': False,
(TaskRunner pid=2074004) 'return_multi_modal_inputs': True,
(TaskRunner pid=2074004) 'return_raw_chat': True,
(TaskRunner pid=2074004) 'return_raw_input_ids': False,
(TaskRunner pid=2074004) 'reward_fn_key': 'data_source',
(TaskRunner pid=2074004) 'sampler': {'class_name': None, 'class_path': None},
(TaskRunner pid=2074004) 'seed': None,
(TaskRunner pid=2074004) 'shuffle': True,
(TaskRunner pid=2074004) 'tokenizer': None,
(TaskRunner pid=2074004) 'tool_config_path': None,
(TaskRunner pid=2074004) 'train_batch_size': 32,
(TaskRunner pid=2074004) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'],
(TaskRunner pid=2074004) 'train_max_samples': 3200,
(TaskRunner pid=2074004) 'truncation': 'error',
(TaskRunner pid=2074004) 'trust_remote_code': True,
(TaskRunner pid=2074004) 'use_shm': False,
(TaskRunner pid=2074004) 'val_batch_size': None,
(TaskRunner pid=2074004) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'],
(TaskRunner pid=2074004) 'val_max_samples': -1,
(TaskRunner pid=2074004) 'validation_shuffle': False,
(TaskRunner pid=2074004) 'video_key': 'videos'},
(TaskRunner pid=2074004) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2074004) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2074004) 'controller_nsight_options': {'cuda-graph-trace': 'graph',
(TaskRunner pid=2074004) 'cuda-memory-usage': 'true',
(TaskRunner pid=2074004) 'trace': 'cuda,nvtx,cublas,ucx'},
(TaskRunner pid=2074004) 'discrete': False,
(TaskRunner pid=2074004) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi',
(TaskRunner pid=2074004) 'capture-range-end': None,
(TaskRunner pid=2074004) 'cuda-graph-trace': 'graph',
(TaskRunner pid=2074004) 'cuda-memory-usage': 'true',
(TaskRunner pid=2074004) 'kill': 'none',
(TaskRunner pid=2074004) 'trace': 'cuda,nvtx,cublas,ucx'}},
(TaskRunner pid=2074004) 'torch_memory': {'context': 'all',
(TaskRunner pid=2074004) 'kw_args': {},
(TaskRunner pid=2074004) 'stack_depth': 32,
(TaskRunner pid=2074004) 'stacks': 'all',
(TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}},
(TaskRunner pid=2074004) 'profile_continuous_steps': False,
(TaskRunner pid=2074004) 'save_path': 'outputs/profile',
(TaskRunner pid=2074004) 'steps': None,
(TaskRunner pid=2074004) 'tool': None},
(TaskRunner pid=2074004) 'max_model_len': 10240,
(TaskRunner pid=2074004) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B',
(TaskRunner pid=2074004) 'include_dashboard': False,
(TaskRunner pid=2074004) 'num_cpus': None,
(TaskRunner pid=2074004) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2074004) 'TASK': 'datasets/sciknoweval/physics',
(TaskRunner pid=2074004) 'USER': 'root'}}},
(TaskRunner pid=2074004) 'timeline_json_file': None},
(TaskRunner pid=2074004) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig',
(TaskRunner pid=2074004) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig',
(TaskRunner pid=2074004) 'name': 'custom_reward_manager',
(TaskRunner pid=2074004) 'path': None},
(TaskRunner pid=2074004) 'name': 'naive',
(TaskRunner pid=2074004) 'source': 'register'},
(TaskRunner pid=2074004) 'reward_model': {'enable': False,
(TaskRunner pid=2074004) 'enable_resource_pool': False,
(TaskRunner pid=2074004) 'forward_max_token_len_per_gpu': 32768,
(TaskRunner pid=2074004) 'launch_reward_fn_async': False,
(TaskRunner pid=2074004) 'max_length': None,
(TaskRunner pid=2074004) 'micro_batch_size': None,
(TaskRunner pid=2074004) 'micro_batch_size_per_gpu': None,
(TaskRunner pid=2074004) 'model': {'external_lib': None,
(TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig',
(TaskRunner pid=2074004) 'forward_prefetch': False,
(TaskRunner pid=2074004) 'fsdp_size': -1,
(TaskRunner pid=2074004) 'param_offload': False,
(TaskRunner pid=2074004) 'reshard_after_forward': True,
(TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}},
(TaskRunner pid=2074004) 'input_tokenizer': 'Qwen/Qwen3-4B',
(TaskRunner pid=2074004) 'override_config': {},
(TaskRunner pid=2074004) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1',
(TaskRunner pid=2074004) 'trust_remote_code': False,
(TaskRunner pid=2074004) 'use_fused_kernels': False,
(TaskRunner pid=2074004) 'use_remove_padding': False,
(TaskRunner pid=2074004) 'use_shm': False},
(TaskRunner pid=2074004) 'n_gpus_per_node': 8,
(TaskRunner pid=2074004) 'nnodes': 0,
(TaskRunner pid=2074004) 'num_workers': 1,
(TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig',
(TaskRunner pid=2074004) 'all_ranks': False,
(TaskRunner pid=2074004) 'enable': False,
(TaskRunner pid=2074004) 'ranks': [],
(TaskRunner pid=2074004) 'save_path': 'outputs/profile',
(TaskRunner pid=2074004) 'tool': None,
(TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig',
(TaskRunner pid=2074004) 'analysis': True,
(TaskRunner pid=2074004) 'contents': [],
(TaskRunner pid=2074004) 'discrete': False,
(TaskRunner pid=2074004) 'level': 'level0'},
(TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig',
(TaskRunner pid=2074004) 'discrete': False},
(TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig',
(TaskRunner pid=2074004) 'step_end': None,
(TaskRunner pid=2074004) 'step_start': 0},
(TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig',
(TaskRunner pid=2074004) 'stack_depth': 32,
(TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}},
(TaskRunner pid=2074004) 'reward_loop_class_name': None,
(TaskRunner pid=2074004) 'reward_loop_module_path': None,
(TaskRunner pid=2074004) 'reward_loop_source': 'register',
(TaskRunner pid=2074004) 'reward_manager': 'naive',
(TaskRunner pid=2074004) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig',
(TaskRunner pid=2074004) 'cudagraph_capture_sizes': None,
(TaskRunner pid=2074004) 'data_parallel_size': 1,
(TaskRunner pid=2074004) 'disable_log_stats': True,
(TaskRunner pid=2074004) 'dtype': 'bfloat16',
(TaskRunner pid=2074004) 'enable_chunked_prefill': True,
(TaskRunner pid=2074004) 'enable_prefix_caching': True,
(TaskRunner pid=2074004) 'enforce_eager': True,
(TaskRunner pid=2074004) 'engine_kwargs': {},
(TaskRunner pid=2074004) 'expert_parallel_size': 1,
(TaskRunner pid=2074004) 'free_cache_engine': True,
(TaskRunner pid=2074004) 'gpu_memory_utilization': 0.5,
(TaskRunner pid=2074004) 'limit_images': None,
(TaskRunner pid=2074004) 'load_format': 'auto',
(TaskRunner pid=2074004) 'max_model_len': None,
(TaskRunner pid=2074004) 'max_num_batched_tokens': 8192,
(TaskRunner pid=2074004) 'max_num_seqs': 1024,
(TaskRunner pid=2074004) 'name': '???',
(TaskRunner pid=2074004) 'prompt_length': 2048,
(TaskRunner pid=2074004) 'response_length': 2048,
(TaskRunner pid=2074004) 'skip_tokenizer_init': False,
(TaskRunner pid=2074004) 'tensor_model_parallel_size': 2},
(TaskRunner pid=2074004) 'sandbox_fusion': {'max_concurrent': 64,
(TaskRunner pid=2074004) 'memory_limit_mb': 1024,
(TaskRunner pid=2074004) 'url': None},
(TaskRunner pid=2074004) 'strategy': 'fsdp',
(TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1,
(TaskRunner pid=2074004) 'use_dynamic_bsz': False,
(TaskRunner pid=2074004) 'use_reward_loop': False},
(TaskRunner pid=2074004) 'trainer': {'balance_batch': True,
(TaskRunner pid=2074004) 'critic_warmup': 0,
(TaskRunner pid=2074004) 'default_hdfs_dir': None,
(TaskRunner pid=2074004) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2074004) 'del_local_ckpt_after_load': False,
(TaskRunner pid=2074004) 'device': 'cuda',
(TaskRunner pid=2074004) 'esi_redundant_time': 0,
(TaskRunner pid=2074004) 'experiment_name': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8',
(TaskRunner pid=2074004) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization',
(TaskRunner pid=2074004) 'log_val_generations': 0,
(TaskRunner pid=2074004) 'logger': ['console', 'wandb'],
(TaskRunner pid=2074004) 'max_actor_ckpt_to_keep': 1,
(TaskRunner pid=2074004) 'max_critic_ckpt_to_keep': None,
(TaskRunner pid=2074004) 'n_gpus_per_node': 8,
(TaskRunner pid=2074004) 'nnodes': 1,
(TaskRunner pid=2074004) 'project_name': 'SDPO-root',
(TaskRunner pid=2074004) 'ray_wait_register_center_timeout': 300,
(TaskRunner pid=2074004) 'resume_from_path': None,
(TaskRunner pid=2074004) 'resume_mode': 'auto',
(TaskRunner pid=2074004) 'rollout_data_dir': None,
(TaskRunner pid=2074004) 'save_freq': 10,
(TaskRunner pid=2074004) 'test_freq': 10,
(TaskRunner pid=2074004) 'total_epochs': 30,
(TaskRunner pid=2074004) 'total_training_steps': 100,
(TaskRunner pid=2074004) 'use_legacy_worker_impl': 'auto',
(TaskRunner pid=2074004) 'val_before_train': False,
(TaskRunner pid=2074004) 'val_only': False,
(TaskRunner pid=2074004) 'validation_data_dir': None},
(TaskRunner pid=2074004) 'transfer_queue': {'enable': False},
(TaskRunner pid=2074004) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics',
(TaskRunner pid=2074004) 'dir': '/users/root/SDPO',
(TaskRunner pid=2074004) 'log_dir': '/users/root/output',
(TaskRunner pid=2074004) 'task': 'datasets/sciknoweval/physics'}}
(TaskRunner pid=2074004) [validate_config] All configuration checks passed successfully!
(TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2074004) use_critic=need_critic(config),
(TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2074004) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(TaskRunner pid=2074004) Using dataset class: RLHFDataset
(TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(TaskRunner pid=2074004) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(TaskRunner pid=2074004) dataset len: 720
(TaskRunner pid=2074004) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2074004) WARNING:2026-07-02 08:56:38,173:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00<?, ? examples/s]
(TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 720/720 [00:00<00:00, 870.16 examples/s]
(TaskRunner pid=2074004) filter dataset len: 720
(TaskRunner pid=2074004) Using dataset class: RLHFDataset
(TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 720/720 [00:00<00:00, 782.95 examples/s]
(TaskRunner pid=2074004) dataset len: 80
(TaskRunner pid=2074004) Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2074004) WARNING:2026-07-02 08:56:39,514:Setting TOKENIZERS_PARALLELISM=false for forked processes.
(TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/80 [00:00<?, ? examples/s]
(TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 151.82 examples/s]
(TaskRunner pid=2074004) filter dataset len: 80
(TaskRunner pid=2074004) Size of train dataloader: 22, Size of val dataloader: 1
(TaskRunner pid=2074004) Total training steps: 100
(TaskRunner pid=2074004) colocated worker base class <class 'verl.single_controller.base.worker.Worker'>
(TaskRunner pid=2074004) bind role actor_rollout_ref method chat_completion to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2074004) bind role actor_rollout_ref method generate to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2074004) bind role actor_rollout_ref method get_zeromq_address to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2074004) bind role actor_rollout_ref method sleep to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2074004) bind role actor_rollout_ref method wake_up to class <class 'verl.single_controller.ray.base.create_colocated_worker_cls.<locals>.WorkerDict'>
(TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 128.23 examples/s]
(TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True
(TaskRunner pid=2074004) self.use_critic = need_critic(self.config)
(WorkerDict pid=2074388) [W702 08:56:47.781903761 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56733 (errno: 97 - Address family not supported by protocol).
(WorkerDict pid=2074385) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7
(WorkerDict pid=2074386) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2074386) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2074387) [W702 08:56:47.990315263 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56733 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
(WorkerDict pid=2074386) `torch_dtype` is deprecated! Use `dtype` instead!
(WorkerDict pid=2074386) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2074386) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)`
(WorkerDict pid=2074391) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]
(WorkerDict pid=2074385) Model config after override: Qwen3Config {
(WorkerDict pid=2074385) "architectures": [
(WorkerDict pid=2074385) "Qwen3ForCausalLM"
(WorkerDict pid=2074385) ],
(WorkerDict pid=2074385) "attention_bias": false,
(WorkerDict pid=2074385) "attention_dropout": 0.0,
(WorkerDict pid=2074385) "dtype": "bfloat16",
(WorkerDict pid=2074385) "eos_token_id": 151645,
(WorkerDict pid=2074385) "head_dim": 128,
(WorkerDict pid=2074385) "hidden_act": "silu",
(WorkerDict pid=2074385) "hidden_size": 2560,
(WorkerDict pid=2074385) "initializer_range": 0.02,
(WorkerDict pid=2074385) "intermediate_size": 9728,
(WorkerDict pid=2074385) "layer_types": [
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention"
(WorkerDict pid=2074385) ],
(WorkerDict pid=2074385) "max_position_embeddings": 40960,
(WorkerDict pid=2074385) "max_window_layers": 36,
(WorkerDict pid=2074385) "model_type": "qwen3",
(WorkerDict pid=2074385) "num_attention_heads": 32,
(WorkerDict pid=2074385) "num_hidden_layers": 36,
(WorkerDict pid=2074385) "num_key_value_heads": 8,
(WorkerDict pid=2074385) "pad_token_id": 151643,
(WorkerDict pid=2074385) "rms_norm_eps": 1e-06,
(WorkerDict pid=2074385) "rope_scaling": null,
(WorkerDict pid=2074385) "rope_theta": 1000000,
(WorkerDict pid=2074385) "sliding_window": null,
(WorkerDict pid=2074385) "tie_word_embeddings": true,
(WorkerDict pid=2074385) "transformers_version": "4.57.1",
(WorkerDict pid=2074385) "use_cache": true,
(WorkerDict pid=2074385) "use_sliding_window": false,
(WorkerDict pid=2074385) "vocab_size": 151936
(WorkerDict pid=2074385) }
(WorkerDict pid=2074385)
(WorkerDict pid=2074387) [Gloo] Rank 2 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 [repeated 7x across cluster]
(WorkerDict pid=2074391) Loading checkpoint shards: 33%|███▎ | 1/3 [00:02<00:05, 2.86s/it]
(WorkerDict pid=2074385) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2074385) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2074385) `torch_dtype` is deprecated! Use `dtype` instead! [repeated 7x across cluster]
(WorkerDict pid=2074385) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` [repeated 14x across cluster]
(WorkerDict pid=2074385) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s] [repeated 7x across cluster]
(WorkerDict pid=2074391) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.91s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it]
(WorkerDict pid=2074388) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it]
(WorkerDict pid=2074391) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention
(WorkerDict pid=2074391) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch
(WorkerDict pid=2074385) Qwen3ForCausalLM contains 4.02B parameters
(WorkerDict pid=2074385) wrap_policy: functools.partial(<function _or_policy at 0x7e4f0b588540>, policies=[functools.partial(<function transformer_auto_wrap_policy at 0x7e4f0b588400>, transformer_layer_cls={<class 'transformers.models.qwen3.modeling_qwen3.Qwen3DecoderLayer'>})])
(WorkerDict pid=2074385) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2074385) Total steps: 100, num_warmup_steps: 10
(WorkerDict pid=2074385) Actor use_remove_padding=True
(WorkerDict pid=2074385) Actor use_fused_kernels=False
(WorkerDict pid=2074385) Actor use_prefix_grouper=False
(WorkerDict pid=2074385) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster]
(WorkerDict pid=2074385) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster]
(WorkerDict pid=2074385) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(WorkerDict pid=2074385) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2074387) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.99s/it] [repeated 12x across cluster]
(WorkerDict pid=2074385) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.90s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 2x across cluster]
(WorkerDict pid=2074387) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.01s/it] [repeated 4x across cluster]
(WorkerDict pid=2074385) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(WorkerDict pid=2074385) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(WorkerDict pid=2074385) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3
(WorkerDict pid=2074386) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster]
(WorkerDict pid=2074386) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2074391) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html .
(WorkerDict pid=2074391) warnings.warn(
(WorkerDict pid=2074385) reference model: Qwen/Qwen3-4B
(WorkerDict pid=2074391) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster]
(WorkerDict pid=2074391) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster]
(WorkerDict pid=2074385) Model config after override: Qwen3Config {
(WorkerDict pid=2074385) "architectures": [
(WorkerDict pid=2074385) "Qwen3ForCausalLM"
(WorkerDict pid=2074385) ],
(WorkerDict pid=2074385) "attention_bias": false,
(WorkerDict pid=2074385) "attention_dropout": 0.0,
(WorkerDict pid=2074385) "dtype": "bfloat16",
(WorkerDict pid=2074385) "eos_token_id": 151645,
(WorkerDict pid=2074385) "head_dim": 128,
(WorkerDict pid=2074385) "hidden_act": "silu",
(WorkerDict pid=2074385) "hidden_size": 2560,
(WorkerDict pid=2074385) "initializer_range": 0.02,
(WorkerDict pid=2074385) "intermediate_size": 9728,
(WorkerDict pid=2074385) "layer_types": [
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention",
(WorkerDict pid=2074385) "full_attention"
(WorkerDict pid=2074385) ],
(WorkerDict pid=2074385) "max_position_embeddings": 40960,
(WorkerDict pid=2074385) "max_window_layers": 36,
(WorkerDict pid=2074385) "model_type": "qwen3",
(WorkerDict pid=2074385) "num_attention_heads": 32,
(WorkerDict pid=2074385) "num_hidden_layers": 36,
(WorkerDict pid=2074385) "num_key_value_heads": 8,
(WorkerDict pid=2074385) "pad_token_id": 151643,
(WorkerDict pid=2074385) "rms_norm_eps": 1e-06,
(WorkerDict pid=2074385) "rope_scaling": null,
(WorkerDict pid=2074385) "rope_theta": 1000000,
(WorkerDict pid=2074385) "sliding_window": null,
(WorkerDict pid=2074385) "tie_word_embeddings": true,
(WorkerDict pid=2074385) "transformers_version": "4.57.1",
(WorkerDict pid=2074385) "use_cache": true,
(WorkerDict pid=2074385) "use_sliding_window": false,
(WorkerDict pid=2074385) "vocab_size": 151936
(WorkerDict pid=2074385) }
(WorkerDict pid=2074385)
(WorkerDict pid=2074387) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster]
(WorkerDict pid=2074391) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]
(WorkerDict pid=2074391) Loading checkpoint shards: 33%|███▎ | 1/3 [00:02<00:05, 2.86s/it]
(WorkerDict pid=2074387) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster]
(WorkerDict pid=2074387) warnings.warn( [repeated 7x across cluster]
(WorkerDict pid=2074387) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(WorkerDict pid=2074387) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(WorkerDict pid=2074392) Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s] [repeated 7x across cluster]
(WorkerDict pid=2074391) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it]
(WorkerDict pid=2074388) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.91s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.96s/it]
(WorkerDict pid=2074391) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention
(WorkerDict pid=2074391) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch
(WorkerDict pid=2074385) Qwen3ForCausalLM contains 4.02B parameters
(WorkerDict pid=2074385) wrap_policy: functools.partial(<function _or_policy at 0x7e4f0b588540>, policies=[functools.partial(<function transformer_auto_wrap_policy at 0x7e4f0b588400>, transformer_layer_cls={<class 'transformers.models.qwen3.modeling_qwen3.Qwen3DecoderLayer'>})])
(WorkerDict pid=2074385) Ref use_remove_padding=True
(WorkerDict pid=2074385) Ref use_fused_kernels=False
(WorkerDict pid=2074385) Ref use_prefix_grouper=False
(WorkerDict pid=2074392) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster]
(WorkerDict pid=2074392) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster]
(TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now.
(TaskRunner pid=2074004) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1)
(WorkerDict pid=2074392) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:06<00:03, 3.02s/it] [repeated 14x across cluster]
(WorkerDict pid=2074392) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.04s/it] [repeated 6x across cluster]
(vLLMHttpServer pid=2075282) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(vLLMHttpServer pid=2075282) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(vLLMHttpServer pid=2075282) ['serve',
(vLLMHttpServer pid=2075282) 'Qwen/Qwen3-4B',
(vLLMHttpServer pid=2075282) '--dtype',
(vLLMHttpServer pid=2075282) 'bfloat16',
(vLLMHttpServer pid=2075282) '--load_format',
(vLLMHttpServer pid=2075282) 'dummy',
(vLLMHttpServer pid=2075282) '--trust_remote_code',
(vLLMHttpServer pid=2075282) '--max_model_len',
(vLLMHttpServer pid=2075282) '40960',
(vLLMHttpServer pid=2075282) '--max_num_seqs',
(vLLMHttpServer pid=2075282) '1024',
(vLLMHttpServer pid=2075282) '--enable_chunked_prefill',
(vLLMHttpServer pid=2075282) '--max_num_batched_tokens',
(vLLMHttpServer pid=2075282) '10240',
(vLLMHttpServer pid=2075282) '--enable_prefix_caching',
(vLLMHttpServer pid=2075282) '--enable_sleep_mode',
(vLLMHttpServer pid=2075282) '--logprobs_mode',
(vLLMHttpServer pid=2075282) 'processed_logprobs',
(vLLMHttpServer pid=2075282) '--disable_custom_all_reduce',
(vLLMHttpServer pid=2075282) '--gpu_memory_utilization',
(vLLMHttpServer pid=2075282) '0.8',
(vLLMHttpServer pid=2075282) '--disable_log_stats',
(vLLMHttpServer pid=2075282) '--tensor_parallel_size',
(vLLMHttpServer pid=2075282) '2',
(vLLMHttpServer pid=2075282) '--seed',
(vLLMHttpServer pid=2075282) '0',
(vLLMHttpServer pid=2075282) '--override_generation_config',
(vLLMHttpServer pid=2075282) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, '
(vLLMHttpServer pid=2075282) '"max_new_tokens": 8192}',
(vLLMHttpServer pid=2075282) '--hf_overrides',
(vLLMHttpServer pid=2075282) '{}',
(vLLMHttpServer pid=2075282) '--scheduling_policy',
(vLLMHttpServer pid=2075282) 'fcfs']
(vLLMHttpServer pid=2075282) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead.
(vLLMHttpServer pid=2075282) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(vLLMHttpServer pid=2075282) WARNING 07-02 08:57:45 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned
(WorkerDict pid=2074385) WARNING 07-02 08:57:52 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'.
(vLLMHttpServer pid=2075287) WARNING 07-02 08:57:46 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster]
(WorkerDict pid=2074389) NCCL version 2.27.5+cuda12.9
(WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph
(WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last):
(WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates
(WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f:
(WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/aotautograd/aqwmczjqsm4pvwfimkpe3rswyj2xfwbwedbijxvwd7tfo5n2eqnk/.2074388.133380852872896.tmp'
(vLLMHttpServer pid=2075287) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster]
(vLLMHttpServer pid=2075287) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster]
(vLLMHttpServer pid=2075287) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster]
(vLLMHttpServer pid=2075287) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster]
(WorkerDict pid=2074385) 2026-07-02 08:58:08,146 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ...
(WorkerDict pid=2074386) 2026-07-02 08:58:08,165 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 4%|▍ | 2/51 [00:00<00:02, 18.43it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 8%|▊ | 4/51 [00:00<00:02, 18.29it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 12%|█▏ | 6/51 [00:00<00:02, 17.75it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 16%|█▌ | 8/51 [00:00<00:02, 18.20it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 20%|█▉ | 10/51 [00:00<00:02, 18.30it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 24%|██▎ | 12/51 [00:00<00:02, 18.31it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 27%|██▋ | 14/51 [00:00<00:02, 18.01it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 31%|███▏ | 16/51 [00:00<00:01, 17.66it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 35%|███▌ | 18/51 [00:01<00:01, 17.20it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 39%|███▉ | 20/51 [00:01<00:01, 17.16it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 43%|████▎ | 22/51 [00:01<00:01, 16.80it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 47%|████▋ | 24/51 [00:01<00:01, 16.60it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 51%|█████ | 26/51 [00:01<00:01, 16.26it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 55%|█████▍ | 28/51 [00:01<00:01, 16.23it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 59%|█████▉ | 30/51 [00:01<00:01, 16.12it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 63%|██████▎ | 32/51 [00:01<00:01, 15.88it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 67%|██████▋ | 34/51 [00:02<00:01, 15.47it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 71%|███████ | 36/51 [00:02<00:00, 15.24it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 75%|███████▍ | 38/51 [00:02<00:00, 15.08it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 78%|███████▊ | 40/51 [00:02<00:00, 13.84it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 82%|████████▏ | 42/51 [00:02<00:00, 13.74it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 86%|████████▋ | 44/51 [00:02<00:00, 14.00it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 90%|█████████ | 46/51 [00:02<00:00, 14.23it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 94%|█████████▍| 48/51 [00:03<00:00, 12.33it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 98%|█████████▊| 50/51 [00:03<00:00, 13.38it/s] Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 15.47it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 0%| | 0/51 [00:00<?, ?it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 6%|▌ | 3/51 [00:00<00:02, 22.87it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 12%|█▏ | 6/51 [00:00<00:01, 23.18it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 18%|█▊ | 9/51 [00:00<00:01, 23.38it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 24%|██▎ | 12/51 [00:00<00:01, 23.39it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 29%|██▉ | 15/51 [00:00<00:01, 23.47it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 35%|███▌ | 18/51 [00:00<00:01, 23.39it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 41%|████ | 21/51 [00:00<00:01, 22.78it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 47%|████▋ | 24/51 [00:01<00:01, 23.04it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 53%|█████▎ | 27/51 [00:01<00:01, 22.96it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 59%|█████▉ | 30/51 [00:01<00:00, 22.77it/s]
(WorkerDict pid=2074387) 2026-07-02 08:58:08,564 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... [repeated 7x across cluster]
(WorkerDict pid=2074387) 2026-07-02 08:58:08,597 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends [repeated 7x across cluster]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 65%|██████▍ | 33/51 [00:01<00:00, 22.44it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 71%|███████ | 36/51 [00:01<00:00, 22.46it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 76%|███████▋ | 39/51 [00:01<00:00, 22.17it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 82%|████████▏ | 42/51 [00:01<00:00, 22.57it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 88%|████████▊ | 45/51 [00:01<00:00, 22.63it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 94%|█████████▍| 48/51 [00:02<00:00, 22.66it/s]
(WorkerDict pid=2074385) Capturing CUDA graphs (decode, FULL): 100%|██████████| 51/51 [00:02<00:00, 22.85it/s] Capturing CUDA graphs (decode, FULL): 100%|██████████| 51/51 [00:02<00:00, 22.84it/s]
(vLLMHttpServer pid=2075282) WARNING 07-02 08:58:16 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development!
(WorkerDict pid=2074387) WARNING 07-02 08:57:54 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. [repeated 7x across cluster]
(WorkerDict pid=2074387) NCCL version 2.27.5+cuda12.9 [repeated 2x across cluster]
(vLLMHttpServer pid=2075282) WARNING 07-02 08:58:16 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`.
(TaskRunner pid=2074004) AgentLoopManager: ['198.19.44.212:45101', '198.19.44.212:38729', '198.19.44.212:33235', '198.19.44.212:45557']
(TaskRunner pid=2074004) wandb: Currently logged in as: seongryongjung (seongryongjung-chung-ang-university) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
(TaskRunner pid=2074004) wandb: setting up run eajdt9hv
(TaskRunner pid=2074004) wandb: Tracking run with wandb version 0.23.1
(TaskRunner pid=2074004) wandb: Run data is saved locally in /mnt/mole/SDPO/L2T/wandb/run-20260702_085822-eajdt9hv
(TaskRunner pid=2074004) wandb: Run `wandb offline` to turn off syncing.
(TaskRunner pid=2074004) wandb: Syncing run qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8
(TaskRunner pid=2074004) wandb: ⭐️ View project at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root
(TaskRunner pid=2074004) wandb: 🚀 View run at https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/eajdt9hv
(TaskRunner pid=2074004) Checkpoint tracker file does not exist: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/latest_checkpointed_iteration.txt
(TaskRunner pid=2074004) Training from scratch
(vLLMHttpServer pid=2075287) WARNING 07-02 08:58:17 [api_server.py:1358] LoRA dynamic loading & unloading is enabled in the API server. This should ONLY be used for local development! [repeated 3x across cluster]
(vLLMHttpServer pid=2075287) WARNING 07-02 08:58:17 [model.py:1576] Default sampling parameters have been overridden by the model's Hugging Face generation config recommended from the model creator. If this is not intended, please relaunch vLLM instance with `--generation-config vllm`. [repeated 3x across cluster]
(TaskRunner pid=2074004) wandb: Detected [openai] in use.
(TaskRunner pid=2074004) wandb: Use W&B Weave for improved LLM call tracing. Install Weave with `pip install weave` then add `import weave` to the top of your script.
(TaskRunner pid=2074004) wandb: For more information, check out the docs at: https://weave-docs.wandb.ai/
(TaskRunner pid=2074004) Training Progress: 0%| | 0/100 [00:00<?, ?it/s]
(AgentLoopWorker pid=2076238) Using dataset class: RLHFDataset
(AgentLoopWorker pid=2076236) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing
(AgentLoopWorker pid=2076236) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1)
(AgentLoopWorker pid=2076238) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
(AgentLoopWorker pid=2076239) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster]
(AgentLoopWorker pid=2076239) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster]
(TaskRunner pid=2074004) step:1 - global_seqlen/min:18309 - global_seqlen/max:26933 - global_seqlen/minmax_diff:8624 - global_seqlen/balanced_min:22775 - global_seqlen/balanced_max:22795 - global_seqlen/mean:22791.5 - actor/entropy:0.12901780009269714 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33952924609184265 - training/rollout_probs_diff_mean:0.003626252058893442 - training/rollout_probs_diff_std:0.013260331004858017 - training/rollout_actor_probs_pearson_corr:0.997394323348999 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.6484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:0.9998406767845154 - rollout_corr/rollout_is_ratio_fraction_high:2.697938725759741e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013489693810697645 - rollout_corr/rollout_is_max:2.4144582748413086 - rollout_corr/rollout_is_min:0.2719356119632721 - rollout_corr/rollout_is_std:0.03670400753617287 - rollout_corr/rollout_is_eff_sample_size:0.9986543905048518 - rollout_corr/rollout_is_seq_mean:0.9997405409812927 - rollout_corr/rollout_is_seq_std:0.002352629555389285 - rollout_corr/rollout_is_seq_max:1.007424235343933 - rollout_corr/rollout_is_seq_min:0.9922156929969788 - rollout_corr/rollout_is_seq_max_deviation:0.00778430700302124 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1334441313520074) - rollout_corr/training_log_ppl:np.float64(0.12146686581036192) - rollout_corr/kl:np.float64(0.0010829019718414656) - rollout_corr/k3_kl:np.float64(0.0008234437515568516) - rollout_corr/rollout_ppl:np.float64(1.132208516355604) - rollout_corr/rollout_log_ppl:np.float64(0.1203839630325092) - rollout_corr/log_ppl_diff:np.float64(0.0010829027778527234) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019076470453001093) - rollout_corr/log_ppl_diff_max:np.float64(0.009199023246765137) - rollout_corr/log_ppl_diff_min:np.float64(-0.0055696964263916016) - rollout_corr/ppl_ratio:np.float64(1.0010864122305065) - rollout_corr/chi2_token:np.float64(0.0011341895442456007) - rollout_corr/chi2_seq:np.float64(1.1553889391943812) - actor/pg_loss:np.float64(0.0009169982513412833) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0) - actor/ppo_kl:np.float64(0.0) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_w_mean:np.float64(101935.92692394112) - self_distillation/token_reweight_w_std:np.float64(1944842.722632798) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.002856640610844) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060511881674756296) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.6032922193408012) - perf/mfu/actor:np.float64(0.03187354289216835) - perf/max_memory_allocated_gb:np.float64(117.27433395385742) - perf/max_memory_reserved_gb:np.float64(122.33203125) - perf/cpu_memory_used_gb:np.float64(153.55620193481445) - actor/lr:np.float64(0.0) - training/global_step:1 - training/epoch:0 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011229045689105988 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011229045689105988 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:434.359375 - response_length/max:136
(AgentLoopWorker pid=2076241) Using dataset class: RLHFDataset [repeated 7x across cluster]
(TaskRunner pid=2074004) Training Progress: 1%| | 1/100 [01:02<1:42:29, 62.12s/it]
(AgentLoopWorker pid=2076235) You're using a Qwen2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding. [repeated 7x across cluster]
(TaskRunner pid=2074004) step:2 - global_seqlen/min:17240 - global_seqlen/max:28034 - global_seqlen/minmax_diff:10794 - global_seqlen/balanced_min:23468 - global_seqlen/balanced_max:23486 - global_seqlen/mean:23480.5 - actor/entropy:0.11835150420665741 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45647841691970825 - training/rollout_probs_diff_mean:0.003378116525709629 - training/rollout_probs_diff_std:0.01276073046028614 - training/rollout_actor_probs_pearson_corr:0.997373640537262 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.765625 - self_distillation/correct_sample_fraction:0.58984375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.765625 - rollout_corr/rollout_is_mean:0.9998647570610046 - rollout_corr/rollout_is_ratio_fraction_high:2.5151748559437692e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014252658002078533 - rollout_corr/rollout_is_max:2.4610891342163086 - rollout_corr/rollout_is_min:0.022991806268692017 - rollout_corr/rollout_is_std:0.03536754101514816 - rollout_corr/rollout_is_eff_sample_size:0.9987503428937182 - rollout_corr/rollout_is_seq_mean:0.9999184012413025 - rollout_corr/rollout_is_seq_std:0.002626801375299692 - rollout_corr/rollout_is_seq_max:1.014352798461914 - rollout_corr/rollout_is_seq_min:0.9884647130966187 - rollout_corr/rollout_is_seq_max_deviation:0.014352798461914062 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1217696573585272) - rollout_corr/training_log_ppl:np.float64(0.11217611807660433) - rollout_corr/kl:np.float64(0.0006482104964864455) - rollout_corr/k3_kl:np.float64(0.0007506794872540468) - rollout_corr/rollout_ppl:np.float64(1.1210116581059992) - rollout_corr/rollout_log_ppl:np.float64(0.11152790688356617) - rollout_corr/log_ppl_diff:np.float64(0.0006482111930381507) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017341631464660168) - rollout_corr/log_ppl_diff_max:np.float64(0.012182682752609253) - rollout_corr/log_ppl_diff_min:np.float64(-0.010948874056339264) - rollout_corr/ppl_ratio:np.float64(1.0006515292916447) - rollout_corr/chi2_token:np.float64(0.0016948485281318426) - rollout_corr/chi2_seq:np.float64(1.00182099849917) - actor/pg_loss:np.float64(0.0020551462657749653) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004427585834037018) - actor/ppo_kl:np.float64(-0.0001974607905879111) - actor/pg_clipfrac_lower:np.float64(1.0866235697903903e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.765625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.765625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_w_mean:np.float64(82757.56090946938) - self_distillation/token_reweight_w_std:np.float64(1471258.8806334345) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015362151898444) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03413831765647046) - self_distillation/empty_target_batch:np.float64(0.234375) - actor/grad_norm:np.float64(0.30624061077833176) - perf/mfu/actor:np.float64(0.04193002600294579) - perf/max_memory_allocated_gb:np.float64(117.33927488327026) - perf/max_memory_reserved_gb:np.float64(122.33203125) - perf/cpu_memory_used_gb:np.float64(149.6733283996582) - actor/lr:np.float64(1e-07) - training/global_step:2 - training/epoch:0 - critic/score/mean:0.58984375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58984375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0032204717863351107 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0032204717863351107 - critic/returns/max:0.875 - critic/retu
(TaskRunner pid=2074004) Training Progress: 2%|▏ | 2/100 [01:38<1:16:15, 46.69s/it]
(TaskRunner pid=2074004) step:3 - global_seqlen/min:16921 - global_seqlen/max:30690 - global_seqlen/minmax_diff:13769 - global_seqlen/balanced_min:23161 - global_seqlen/balanced_max:23173 - global_seqlen/mean:23168.875 - actor/entropy:0.13500291109085083 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3997553586959839 - training/rollout_probs_diff_mean:0.0036152575630694628 - training/rollout_probs_diff_std:0.012815049849450588 - training/rollout_actor_probs_pearson_corr:0.9976134300231934 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9998619556427002 - rollout_corr/rollout_is_ratio_fraction_high:2.5933833967428654e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012966917711310089 - rollout_corr/rollout_is_max:2.25616192817688 - rollout_corr/rollout_is_min:0.060227103531360626 - rollout_corr/rollout_is_std:0.03635154664516449 - rollout_corr/rollout_is_eff_sample_size:0.9986798333320833 - rollout_corr/rollout_is_seq_mean:0.999671995639801 - rollout_corr/rollout_is_seq_std:0.003052868414670229 - rollout_corr/rollout_is_seq_max:1.0084736347198486 - rollout_corr/rollout_is_seq_min:0.978779673576355 - rollout_corr/rollout_is_seq_max_deviation:0.02122032642364502 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1375840730033815) - rollout_corr/training_log_ppl:np.float64(0.1260819717645063) - rollout_corr/kl:np.float64(0.0014236645348155719) - rollout_corr/k3_kl:np.float64(0.0009659073380809957) - rollout_corr/rollout_ppl:np.float64(1.135932678822428) - rollout_corr/rollout_log_ppl:np.float64(0.12465830726432614) - rollout_corr/log_ppl_diff:np.float64(0.0014236645001801662) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022198438164195977) - rollout_corr/log_ppl_diff_max:np.float64(0.044630393385887146) - rollout_corr/log_ppl_diff_min:np.float64(-0.0061286017298698425) - rollout_corr/ppl_ratio:np.float64(1.001434272620827) - rollout_corr/chi2_token:np.float64(0.0007382424082607031) - rollout_corr/chi2_seq:np.float64(0.3367840179707855) - actor/pg_loss:np.float64(0.0039574692491441965) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005755162846980966) - actor/ppo_kl:np.float64(0.0001336684559625212) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(138857.04420319246) - self_distillation/token_reweight_w_std:np.float64(1861952.1508011823) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008748008403927) - self_distillation/token_reweight_w_clip_frac:np.float64(0.059623488574288785) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.7043501883745193) - perf/mfu/actor:np.float64(0.041711428711565315) - perf/max_memory_allocated_gb:np.float64(117.41665840148926) - perf/max_memory_reserved_gb:np.float64(122.33203125) - perf/cpu_memory_used_gb:np.float64(147.72043561935425) - actor/lr:np.float64(2e-07) - training/global_step:3 - training/epoch:0 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0074819112196564674 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0074819112196564674 - critic/returns/max:0.875 - critic/returns/min:-
(TaskRunner pid=2074004) Training Progress: 3%|▎ | 3/100 [02:12<1:06:33, 41.17s/it]
(TaskRunner pid=2074004) step:4 - global_seqlen/min:16405 - global_seqlen/max:31747 - global_seqlen/minmax_diff:15342 - global_seqlen/balanced_min:22559 - global_seqlen/balanced_max:22569 - global_seqlen/mean:22566.125 - actor/entropy:0.12252338975667953 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4399338960647583 - training/rollout_probs_diff_mean:0.0035380814224481583 - training/rollout_probs_diff_std:0.013156313449144363 - training/rollout_actor_probs_pearson_corr:0.9972266554832458 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.921875 - self_distillation/correct_sample_fraction:0.6875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.921875 - rollout_corr/rollout_is_mean:0.999912440776825 - rollout_corr/rollout_is_ratio_fraction_high:7.157107756938785e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.84102298389189e-05 - rollout_corr/rollout_is_max:3.9813621044158936 - rollout_corr/rollout_is_min:0.2154025137424469 - rollout_corr/rollout_is_std:0.03721042722463608 - rollout_corr/rollout_is_eff_sample_size:0.9986170608525367 - rollout_corr/rollout_is_seq_mean:0.9998709559440613 - rollout_corr/rollout_is_seq_std:0.0019483763026073575 - rollout_corr/rollout_is_seq_max:1.0068435668945312 - rollout_corr/rollout_is_seq_min:0.9933158159255981 - rollout_corr/rollout_is_seq_max_deviation:0.00684356689453125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1197912953794003) - rollout_corr/training_log_ppl:np.float64(0.11032333969706087) - rollout_corr/kl:np.float64(0.000864178637373314) - rollout_corr/k3_kl:np.float64(0.0007223768616420045) - rollout_corr/rollout_ppl:np.float64(1.1188077311962843) - rollout_corr/rollout_log_ppl:np.float64(0.10945915975025855) - rollout_corr/log_ppl_diff:np.float64(0.0008641799468023237) - rollout_corr/log_ppl_abs_diff:np.float64(0.001754023334797239) - rollout_corr/log_ppl_diff_max:np.float64(0.010782986879348755) - rollout_corr/log_ppl_diff_min:np.float64(-0.004266001284122467) - rollout_corr/ppl_ratio:np.float64(1.000867168419063) - rollout_corr/chi2_token:np.float64(0.0011759297922253609) - rollout_corr/chi2_seq:np.float64(0.552473843563348) - actor/pg_loss:np.float64(0.003973975777626038) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008309469519645063) - actor/ppo_kl:np.float64(2.3139348304113128e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.921875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.921875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_w_mean:np.float64(120593.3020443772) - self_distillation/token_reweight_w_std:np.float64(2251003.20675789) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000611402792856) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06828094898082782) - self_distillation/empty_target_batch:np.float64(0.078125) - actor/grad_norm:np.float64(0.3640708401799202) - perf/mfu/actor:np.float64(0.040760511232844326) - perf/max_memory_allocated_gb:np.float64(117.41665840148926) - perf/max_memory_reserved_gb:np.float64(122.33203125) - perf/cpu_memory_used_gb:np.float64(146.10588455200195) - actor/lr:np.float64(3e-07) - training/global_step:4 - training/epoch:0 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.008499065414071083 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.008499065414071083 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:436.6289
(TaskRunner pid=2074004) Training Progress: 4%|▍ | 4/100 [02:46<1:01:29, 38.44s/it]
(TaskRunner pid=2074004) step:5 - global_seqlen/min:20128 - global_seqlen/max:33499 - global_seqlen/minmax_diff:13371 - global_seqlen/balanced_min:23612 - global_seqlen/balanced_max:29654 - global_seqlen/mean:24388.125 - actor/entropy:0.09556688368320465 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4399767816066742 - training/rollout_probs_diff_mean:0.002684555947780609 - training/rollout_probs_diff_std:0.0111756082624197 - training/rollout_actor_probs_pearson_corr:0.9976402521133423 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.90234375 - self_distillation/correct_sample_fraction:0.7578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.90234375 - rollout_corr/rollout_is_mean:1.0000108480453491 - rollout_corr/rollout_is_ratio_fraction_high:1.5412000720971264e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.476600487483665e-05 - rollout_corr/rollout_is_max:2.115692138671875 - rollout_corr/rollout_is_min:0.2046172171831131 - rollout_corr/rollout_is_std:0.031114283949136734 - rollout_corr/rollout_is_eff_sample_size:0.9990327186892993 - rollout_corr/rollout_is_seq_mean:0.999940037727356 - rollout_corr/rollout_is_seq_std:0.0020009703002870083 - rollout_corr/rollout_is_seq_max:1.0067591667175293 - rollout_corr/rollout_is_seq_min:0.9904017448425293 - rollout_corr/rollout_is_seq_max_deviation:0.009598255157470703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.098601276986301) - rollout_corr/training_log_ppl:np.float64(0.0921296329697725) - rollout_corr/kl:np.float64(0.0007925959318022535) - rollout_corr/k3_kl:np.float64(0.0006387021954168404) - rollout_corr/rollout_ppl:np.float64(1.0977234644815326) - rollout_corr/rollout_log_ppl:np.float64(0.09133703709630936) - rollout_corr/log_ppl_diff:np.float64(0.0007925958734631422) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015993376027836348) - rollout_corr/log_ppl_diff_max:np.float64(0.010799288749694824) - rollout_corr/log_ppl_diff_min:np.float64(-0.005030497908592224) - rollout_corr/ppl_ratio:np.float64(1.000795227708295) - rollout_corr/chi2_token:np.float64(0.000898352125659585) - rollout_corr/chi2_seq:np.float64(0.32050589215941727) - actor/pg_loss:np.float64(0.001984496833756566) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004709874367563316) - actor/ppo_kl:np.float64(0.00010625433543509644) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.90234375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.90234375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7578125) - self_distillation/token_reweight_w_mean:np.float64(118120.8138428235) - self_distillation/token_reweight_w_std:np.float64(2024403.102559087) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000120525015518) - self_distillation/token_reweight_w_clip_frac:np.float64(0.032204644754529) - self_distillation/empty_target_batch:np.float64(0.09765625) - actor/grad_norm:np.float64(0.2131564086303115) - perf/mfu/actor:np.float64(0.04161984583528429) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(142.10017776489258) - actor/lr:np.float64(4e-07) - training/global_step:5 - training/epoch:0 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011104347184300423 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011104347184300423 - critic/returns/max:0.875 - critic/returns/min:-0.875 - respons
(TaskRunner pid=2074004) Training Progress: 5%|▌ | 5/100 [04:03<1:22:52, 52.35s/it]
(TaskRunner pid=2074004) step:6 - global_seqlen/min:19327 - global_seqlen/max:36523 - global_seqlen/minmax_diff:17196 - global_seqlen/balanced_min:25576 - global_seqlen/balanced_max:25603 - global_seqlen/mean:25591.375 - actor/entropy:0.1292387694120407 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5271100401878357 - training/rollout_probs_diff_mean:0.003469154005870223 - training/rollout_probs_diff_std:0.012764371000230312 - training/rollout_actor_probs_pearson_corr:0.9976286888122559 - self_distillation/success_group_fraction:0.9375 - self_distillation/success_sample_fraction:0.91796875 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.91796875 - rollout_corr/rollout_is_mean:1.0000030994415283 - rollout_corr/rollout_is_ratio_fraction_high:3.0438852263614535e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.13165567908436e-05 - rollout_corr/rollout_is_max:3.314143657684326 - rollout_corr/rollout_is_min:0.0985947847366333 - rollout_corr/rollout_is_std:0.0357680581510067 - rollout_corr/rollout_is_eff_sample_size:0.9987223994650055 - rollout_corr/rollout_is_seq_mean:1.0000648498535156 - rollout_corr/rollout_is_seq_std:0.0021835637744516134 - rollout_corr/rollout_is_seq_max:1.008790373802185 - rollout_corr/rollout_is_seq_min:0.9911096096038818 - rollout_corr/rollout_is_seq_max_deviation:0.008890390396118164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.144010590389371) - rollout_corr/training_log_ppl:np.float64(0.1311971230970812) - rollout_corr/kl:np.float64(0.0006381959663634973) - rollout_corr/k3_kl:np.float64(0.0008073146158267264) - rollout_corr/rollout_ppl:np.float64(1.1432772078551352) - rollout_corr/rollout_log_ppl:np.float64(0.13055892662669066) - rollout_corr/log_ppl_diff:np.float64(0.0006381964703905396) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017149751001852565) - rollout_corr/log_ppl_diff_max:np.float64(0.0076720863580703735) - rollout_corr/log_ppl_diff_min:np.float64(-0.010425284504890442) - rollout_corr/ppl_ratio:np.float64(1.000641003018245) - rollout_corr/chi2_token:np.float64(0.002027132548391819) - rollout_corr/chi2_seq:np.float64(0.9608263978734612) - actor/pg_loss:np.float64(0.004716917057521641) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007475859654277883) - actor/ppo_kl:np.float64(-5.642967614116756e-05) - actor/pg_clipfrac_lower:np.float64(3.379109102752409e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.91796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.91796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(100609.92611906817) - self_distillation/token_reweight_w_std:np.float64(1991495.2712052793) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0024336345959455) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07567856350215152) - self_distillation/empty_target_batch:np.float64(0.08203125) - actor/grad_norm:np.float64(0.39267846941947937) - perf/mfu/actor:np.float64(0.04579480447676534) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(141.87373733520508) - actor/lr:np.float64(5e-07) - training/global_step:6 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005125141702592373 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005125141702592373 - critic/returns/max:0.875 - critic/ret
(TaskRunner pid=2074004) Training Progress: 6%|▌ | 6/100 [04:40<1:13:47, 47.10s/it]
(TaskRunner pid=2074004) step:7 - global_seqlen/min:17223 - global_seqlen/max:29309 - global_seqlen/minmax_diff:12086 - global_seqlen/balanced_min:21951 - global_seqlen/balanced_max:21973 - global_seqlen/mean:21966.0 - actor/entropy:0.1024952083826065 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3915199041366577 - training/rollout_probs_diff_mean:0.0032435795292258263 - training/rollout_probs_diff_std:0.013019987381994724 - training/rollout_actor_probs_pearson_corr:0.9969437122344971 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:0.99991375207901 - rollout_corr/rollout_is_ratio_fraction_high:4.694306699093431e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010327474592486396 - rollout_corr/rollout_is_max:3.273437261581421 - rollout_corr/rollout_is_min:0.3346708416938782 - rollout_corr/rollout_is_std:0.03565957769751549 - rollout_corr/rollout_is_eff_sample_size:0.9987298905286599 - rollout_corr/rollout_is_seq_mean:0.9998611211776733 - rollout_corr/rollout_is_seq_std:0.0023592528887093067 - rollout_corr/rollout_is_seq_max:1.008383870124817 - rollout_corr/rollout_is_seq_min:0.9914498329162598 - rollout_corr/rollout_is_seq_max_deviation:0.008550167083740234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1064550378359854) - rollout_corr/training_log_ppl:np.float64(0.09936254937929334) - rollout_corr/kl:np.float64(0.0009681725461949675) - rollout_corr/k3_kl:np.float64(0.000758711741880802) - rollout_corr/rollout_ppl:np.float64(1.1053752973675728) - rollout_corr/rollout_log_ppl:np.float64(0.09839437552000163) - rollout_corr/log_ppl_diff:np.float64(0.00096817385929171) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018474093376426026) - rollout_corr/log_ppl_diff_max:np.float64(0.011777296662330627) - rollout_corr/log_ppl_diff_min:np.float64(-0.007294714450836182) - rollout_corr/ppl_ratio:np.float64(1.0009716332424432) - rollout_corr/chi2_token:np.float64(0.00117649813182652) - rollout_corr/chi2_seq:np.float64(1.213416572427377) - actor/pg_loss:np.float64(0.002533678780309856) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007372280974777823) - actor/ppo_kl:np.float64(0.00011072156120306431) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(172788.5278644862) - self_distillation/token_reweight_w_std:np.float64(2824136.8971029455) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999886873876676) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04149601675453596) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.33883800357580185) - perf/mfu/actor:np.float64(0.03960816235298069) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(141.78826522827148) - actor/lr:np.float64(6e-07) - training/global_step:7 - training/epoch:0 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013935049995779991 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013935049995779991 - critic/returns/max:0.875 - critic/returns/min:-0.875 - respons
(TaskRunner pid=2074004) Training Progress: 7%|▋ | 7/100 [05:15<1:06:53, 43.16s/it]
(TaskRunner pid=2074004) step:8 - global_seqlen/min:21969 - global_seqlen/max:30704 - global_seqlen/minmax_diff:8735 - global_seqlen/balanced_min:24931 - global_seqlen/balanced_max:24949 - global_seqlen/mean:24940.375 - actor/entropy:0.12515239417552948 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46682578325271606 - training/rollout_probs_diff_mean:0.003238941542804241 - training/rollout_probs_diff_std:0.012210790999233723 - training/rollout_actor_probs_pearson_corr:0.9976688623428345 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.875 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.875 - rollout_corr/rollout_is_mean:1.000011920928955 - rollout_corr/rollout_is_ratio_fraction_high:1.5385207007057033e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.923342880327255e-05 - rollout_corr/rollout_is_max:2.0662426948547363 - rollout_corr/rollout_is_min:0.13663901388645172 - rollout_corr/rollout_is_std:0.03388545289635658 - rollout_corr/rollout_is_eff_sample_size:0.9988529740930273 - rollout_corr/rollout_is_seq_mean:0.9999704957008362 - rollout_corr/rollout_is_seq_std:0.0018234980525448918 - rollout_corr/rollout_is_seq_max:1.006734848022461 - rollout_corr/rollout_is_seq_min:0.992646336555481 - rollout_corr/rollout_is_seq_max_deviation:0.007353663444519043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1400793073698878) - rollout_corr/training_log_ppl:np.float64(0.1265813655263628) - rollout_corr/kl:np.float64(0.0006476453545214422) - rollout_corr/k3_kl:np.float64(0.0006364178093605233) - rollout_corr/rollout_ppl:np.float64(1.1393103343434632) - rollout_corr/rollout_log_ppl:np.float64(0.12593371887123794) - rollout_corr/log_ppl_diff:np.float64(0.000647646655124845) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014404457069758791) - rollout_corr/log_ppl_diff_max:np.float64(0.00994350016117096) - rollout_corr/log_ppl_diff_min:np.float64(-0.005098991096019745) - rollout_corr/ppl_ratio:np.float64(1.0006497099529952) - rollout_corr/chi2_token:np.float64(0.0012529646046459675) - rollout_corr/chi2_seq:np.float64(0.9698494633194059) - actor/pg_loss:np.float64(0.0041176737286150455) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000545344151532845) - actor/ppo_kl:np.float64(-1.8346452039352812e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(185696.14268651744) - self_distillation/token_reweight_w_std:np.float64(2811707.8566971384) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001049543498084) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05756850506440969) - self_distillation/empty_target_batch:np.float64(0.125) - actor/grad_norm:np.float64(0.412463016808033) - perf/mfu/actor:np.float64(0.045180137195399296) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(141.8076286315918) - actor/lr:np.float64(7e-07) - training/global_step:8 - training/epoch:0 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014383245259523392 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014383245259523392 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:507.792968
(TaskRunner pid=2074004) Training Progress: 8%|▊ | 8/100 [05:53<1:03:26, 41.38s/it]
(TaskRunner pid=2074004) step:9 - global_seqlen/min:17678 - global_seqlen/max:30777 - global_seqlen/minmax_diff:13099 - global_seqlen/balanced_min:22366 - global_seqlen/balanced_max:22384 - global_seqlen/mean:22379.75 - actor/entropy:0.11243886500597 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5107135772705078 - training/rollout_probs_diff_mean:0.0032545437570661306 - training/rollout_probs_diff_std:0.012385137379169464 - training/rollout_actor_probs_pearson_corr:0.9973602890968323 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.000034213066101 - rollout_corr/rollout_is_ratio_fraction_high:2.6944009732687846e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001167573791462928 - rollout_corr/rollout_is_max:2.374828815460205 - rollout_corr/rollout_is_min:0.3055213987827301 - rollout_corr/rollout_is_std:0.034282419830560684 - rollout_corr/rollout_is_eff_sample_size:0.9988262142157865 - rollout_corr/rollout_is_seq_mean:1.000186800956726 - rollout_corr/rollout_is_seq_std:0.002165162470191717 - rollout_corr/rollout_is_seq_max:1.0093048810958862 - rollout_corr/rollout_is_seq_min:0.9918913245201111 - rollout_corr/rollout_is_seq_max_deviation:0.00930488109588623 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1258266768418252) - rollout_corr/training_log_ppl:np.float64(0.11502090978319757) - rollout_corr/kl:np.float64(0.0005568322052704744) - rollout_corr/k3_kl:np.float64(0.0006635844595450635) - rollout_corr/rollout_ppl:np.float64(1.1251782258041203) - rollout_corr/rollout_log_ppl:np.float64(0.1144640768688987) - rollout_corr/log_ppl_diff:np.float64(0.0005568329142988659) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016462723942822777) - rollout_corr/log_ppl_diff_max:np.float64(0.0058449506759643555) - rollout_corr/log_ppl_diff_min:np.float64(-0.012020483613014221) - rollout_corr/ppl_ratio:np.float64(1.0005593579262495) - rollout_corr/chi2_token:np.float64(0.0015625548548996449) - rollout_corr/chi2_seq:np.float64(0.3493979668710381) - actor/pg_loss:np.float64(0.002061657956801355) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004324964299939893) - actor/ppo_kl:np.float64(8.322711867236876e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(196105.12180940015) - self_distillation/token_reweight_w_std:np.float64(2197213.2855446315) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992129243910313) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04007138988527004) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.3090107664465904) - perf/mfu/actor:np.float64(0.040113429579585386) - perf/max_memory_allocated_gb:np.float64(124.924560546875) - perf/max_memory_reserved_gb:np.float64(131.43359375) - perf/cpu_memory_used_gb:np.float64(141.86093521118164) - actor/lr:np.float64(8e-07) - training/global_step:9 - training/epoch:0 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009928868152201176 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009928868152201176 - critic/returns/max:0.875 - critic/returns/min:-0.875 -
(TaskRunner pid=2074004) Training Progress: 9%|▉ | 9/100 [06:28<59:52, 39.48s/it]
(TaskRunner pid=2074004) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 10}
Reference in New Issue Copy Permalink