[2026-07-02 06:18:50] Starting L2T batch32 Qwen3-4B GRPO/RLSD_TR dataset-first queue [2026-07-02 06:18:50] Commit: ee6a667 [2026-07-02 06:18:50] Settings: model=Qwen/Qwen3-4B steps=100 train_batch=32 rlsd_tr_batch=32 rollout=8 mini_batch=8 tr_mix=0.1 max_model_len=10240 vllm=0.8 [2026-07-02 06:18:50] Skipping chemistry grpo; completed by previous queue [2026-07-02 06:18:50] Starting chemistry rlsd_tr Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: chemistry Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet Ray tmp: /tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B 2026-07-02 06:18:52,310 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/chemistry Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/chemistry +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/chemistry', 'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 06:19:01,985 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2026458) TaskRunner hostname: mole-workspace-rw, PID: 2026458 (TaskRunner pid=2026458) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2026458) 'calculate_entropy': False, (TaskRunner pid=2026458) 'calculate_sum_pi_squared': False, (TaskRunner pid=2026458) 'checkpoint': {'_target_': (TaskRunner pid=2026458) 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2026458) 'async_save': False, (TaskRunner pid=2026458) 'load_contents': ['model', (TaskRunner pid=2026458) 'optimizer', (TaskRunner pid=2026458) 'extra'], (TaskRunner pid=2026458) 'save_contents': ['model', (TaskRunner pid=2026458) 'optimizer', (TaskRunner pid=2026458) 'extra']}, (TaskRunner pid=2026458) 'clip_ratio': 0.2, (TaskRunner pid=2026458) 'clip_ratio_c': 3.0, (TaskRunner pid=2026458) 'clip_ratio_high': 0.28, (TaskRunner pid=2026458) 'clip_ratio_low': 0.2, (TaskRunner pid=2026458) 'data_loader_seed': 42, (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_coeff': 0, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'freeze_vision_tower': False, (TaskRunner pid=2026458) 'fsdp_config': (TaskRunner pid=2026458) {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': False, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'grad_clip': 1.0, (TaskRunner pid=2026458) 'kl_loss_coef': 0.001, (TaskRunner pid=2026458) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2026458) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2026458) 'loss_scale_factor': None, (TaskRunner pid=2026458) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2026458) 'betas': [0.9, 0.999], (TaskRunner pid=2026458) 'clip_grad': 1.0, (TaskRunner pid=2026458) 'lr': 1e-06, (TaskRunner pid=2026458) 'lr_scheduler_type': 'constant', (TaskRunner pid=2026458) 'lr_warmup_steps': 10, (TaskRunner pid=2026458) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2026458) 'min_lr_ratio': 0.0, (TaskRunner pid=2026458) 'num_cycles': 0.5, (TaskRunner pid=2026458) 'optimizer': 'AdamW', (TaskRunner pid=2026458) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2026458) 'override_optimizer_config': None, (TaskRunner pid=2026458) 'total_training_steps': -1, (TaskRunner pid=2026458) 'warmup_style': None, (TaskRunner pid=2026458) 'weight_decay': 0.01}, (TaskRunner pid=2026458) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2026458) 'clip_cov_lb': 1.0, (TaskRunner pid=2026458) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2026458) 'clip_cov_ub': 5.0, (TaskRunner pid=2026458) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2026458) 'loss_mode': 'rlsd', (TaskRunner pid=2026458) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2026458) 'ppo_epochs': 1, (TaskRunner pid=2026458) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'ppo_micro_batch_size': None, (TaskRunner pid=2026458) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'ppo_mini_batch_size': 8, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2026458) 'mode': 'disabled', (TaskRunner pid=2026458) 'record_file': None, (TaskRunner pid=2026458) 'replay_file': None}, (TaskRunner pid=2026458) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2026458) 'alpha': 0.5, (TaskRunner pid=2026458) 'distillation_add_tail': True, (TaskRunner pid=2026458) 'distillation_topk': 100, (TaskRunner pid=2026458) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2026458) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2026458) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'loss_weight': 0.0, (TaskRunner pid=2026458) 'mask': 'all'}, (TaskRunner pid=2026458) 'feedback_template': '\n' (TaskRunner pid=2026458) 'The ' (TaskRunner pid=2026458) 'following ' (TaskRunner pid=2026458) 'is ' (TaskRunner pid=2026458) 'feedback ' (TaskRunner pid=2026458) 'from ' (TaskRunner pid=2026458) 'your ' (TaskRunner pid=2026458) 'unsuccessful ' (TaskRunner pid=2026458) 'earlier ' (TaskRunner pid=2026458) 'attempt:\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) '{feedback_raw}', (TaskRunner pid=2026458) 'full_logit_distillation': True, (TaskRunner pid=2026458) 'include_environment_feedback': True, (TaskRunner pid=2026458) 'is_clip': 2, (TaskRunner pid=2026458) 'max_reprompt_len': 10240, (TaskRunner pid=2026458) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2026458) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) 'Correctly ' (TaskRunner pid=2026458) 'solve ' (TaskRunner pid=2026458) 'the ' (TaskRunner pid=2026458) 'original ' (TaskRunner pid=2026458) 'question.', (TaskRunner pid=2026458) 'reprompt_truncation': 'right', (TaskRunner pid=2026458) 'solution_template': '\n' (TaskRunner pid=2026458) 'Correct ' (TaskRunner pid=2026458) 'solution:\n' (TaskRunner pid=2026458) '\n' (TaskRunner pid=2026458) '{successful_previous_attempt}', (TaskRunner pid=2026458) 'srpo_dynamic_weighting': False, (TaskRunner pid=2026458) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2026458) 'success_reward_threshold': 0.5, (TaskRunner pid=2026458) 'teacher_regularization': 'trust-region', (TaskRunner pid=2026458) 'teacher_update_rate': 0.1, (TaskRunner pid=2026458) 'token_reweight_decay_steps': 0, (TaskRunner pid=2026458) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2026458) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2026458) 'shuffle': False, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2026458) 'tau_neg': 1.05, (TaskRunner pid=2026458) 'tau_pos': 1.0, (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_kl_loss': False, (TaskRunner pid=2026458) 'use_prefix_grouper': False, (TaskRunner pid=2026458) 'use_remove_padding': True, (TaskRunner pid=2026458) 'use_torch_compile': True}, (TaskRunner pid=2026458) 'hybrid_engine': True, (TaskRunner pid=2026458) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2026458) 'custom_chat_template': None, (TaskRunner pid=2026458) 'enable_activation_offload': False, (TaskRunner pid=2026458) 'enable_gradient_checkpointing': True, (TaskRunner pid=2026458) 'exclude_modules': None, (TaskRunner pid=2026458) 'external_lib': None, (TaskRunner pid=2026458) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2026458) 'hf_config_path': None, (TaskRunner pid=2026458) 'lora_adapter_path': None, (TaskRunner pid=2026458) 'lora_alpha': 16, (TaskRunner pid=2026458) 'lora_rank': 0, (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'target_modules': 'all-linear', (TaskRunner pid=2026458) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2026458) 'num_shards': 4}, (TaskRunner pid=2026458) 'tokenizer_path': None, (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_liger': False, (TaskRunner pid=2026458) 'use_remove_padding': True, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'nccl_timeout': 600, (TaskRunner pid=2026458) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': True, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'log_prob_micro_batch_size': None, (TaskRunner pid=2026458) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2026458) 'mode': 'disabled', (TaskRunner pid=2026458) 'record_file': None, (TaskRunner pid=2026458) 'replay_file': None}, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_torch_compile': True}, (TaskRunner pid=2026458) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2026458) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2026458) 'agent_loop_config_path': None, (TaskRunner pid=2026458) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2026458) 'name': None, (TaskRunner pid=2026458) 'path': None}, (TaskRunner pid=2026458) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2026458) 'num_workers': 8}, (TaskRunner pid=2026458) 'calculate_log_probs': True, (TaskRunner pid=2026458) 'cudagraph_capture_sizes': None, (TaskRunner pid=2026458) 'data_parallel_size': 1, (TaskRunner pid=2026458) 'disable_log_stats': True, (TaskRunner pid=2026458) 'do_sample': True, (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'enable_chunked_prefill': True, (TaskRunner pid=2026458) 'enable_prefix_caching': True, (TaskRunner pid=2026458) 'enable_rollout_routing_replay': False, (TaskRunner pid=2026458) 'enforce_eager': False, (TaskRunner pid=2026458) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2026458) 'expert_parallel_size': 1, (TaskRunner pid=2026458) 'free_cache_engine': True, (TaskRunner pid=2026458) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2026458) 'ignore_eos': False, (TaskRunner pid=2026458) 'layered_summon': False, (TaskRunner pid=2026458) 'load_format': 'dummy', (TaskRunner pid=2026458) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2026458) 'log_prob_micro_batch_size': None, (TaskRunner pid=2026458) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2026458) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2026458) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2026458) 'max_model_len': 10240, (TaskRunner pid=2026458) 'max_num_batched_tokens': 10240, (TaskRunner pid=2026458) 'max_num_seqs': 1024, (TaskRunner pid=2026458) 'mode': 'async', (TaskRunner pid=2026458) 'multi_stage_wake_up': False, (TaskRunner pid=2026458) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'format': 'hermes', (TaskRunner pid=2026458) 'interaction_config_path': None, (TaskRunner pid=2026458) 'max_assistant_turns': None, (TaskRunner pid=2026458) 'max_parallel_calls': 1, (TaskRunner pid=2026458) 'max_tool_response_length': 256, (TaskRunner pid=2026458) 'max_user_turns': None, (TaskRunner pid=2026458) 'num_repeat_rollouts': None, (TaskRunner pid=2026458) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2026458) 'tool_config_path': None, (TaskRunner pid=2026458) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2026458) 'use_inference_chat_template': False}, (TaskRunner pid=2026458) 'n': 8, (TaskRunner pid=2026458) 'name': 'vllm', (TaskRunner pid=2026458) 'over_sample_rate': 0, (TaskRunner pid=2026458) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2026458) 'port': 9090, (TaskRunner pid=2026458) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2026458) 'prompt_length': 2048, (TaskRunner pid=2026458) 'quantization': None, (TaskRunner pid=2026458) 'quantization_config_file': None, (TaskRunner pid=2026458) 'response_length': 8192, (TaskRunner pid=2026458) 'scheduling_policy': 'fcfs', (TaskRunner pid=2026458) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2026458) 'skip_rollout': False, (TaskRunner pid=2026458) 'skip_tokenizer_init': True, (TaskRunner pid=2026458) 'temperature': 1.0, (TaskRunner pid=2026458) 'tensor_model_parallel_size': 2, (TaskRunner pid=2026458) 'top_k': -1, (TaskRunner pid=2026458) 'top_p': 1.0, (TaskRunner pid=2026458) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2026458) 'backend': None, (TaskRunner pid=2026458) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2026458) 'token2text': False}, (TaskRunner pid=2026458) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2026458) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2026458) 'do_sample': True, (TaskRunner pid=2026458) 'n': 16, (TaskRunner pid=2026458) 'temperature': 0.6, (TaskRunner pid=2026458) 'top_k': -1, (TaskRunner pid=2026458) 'top_p': 0.95}}}, (TaskRunner pid=2026458) 'algorithm': (TaskRunner pid=2026458) {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2026458) 'adv_estimator': 'grpo', (TaskRunner pid=2026458) 'gamma': 1.0, (TaskRunner pid=2026458) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2026458) 'horizon': 10000, (TaskRunner pid=2026458) 'kl_coef': 0.001, (TaskRunner pid=2026458) 'target_kl': 0.1, (TaskRunner pid=2026458) 'type': 'fixed'}, (TaskRunner pid=2026458) 'kl_penalty': 'kl', (TaskRunner pid=2026458) 'lam': 1.0, (TaskRunner pid=2026458) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2026458) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2026458) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2026458) 'loss_type': 'ppo_clip', (TaskRunner pid=2026458) 'rollout_is': 'token', (TaskRunner pid=2026458) 'rollout_is_batch_normalize': False, (TaskRunner pid=2026458) 'rollout_is_threshold': 2.0, (TaskRunner pid=2026458) 'rollout_rs': None, (TaskRunner pid=2026458) 'rollout_rs_threshold': None}, (TaskRunner pid=2026458) 'use_kl_in_reward': False, (TaskRunner pid=2026458) 'use_pf_ppo': False}, (TaskRunner pid=2026458) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2026458) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2026458) 'async_save': False, (TaskRunner pid=2026458) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2026458) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2026458) 'cliprange_value': 0.5, (TaskRunner pid=2026458) 'data_loader_seed': 42, (TaskRunner pid=2026458) 'enable': None, (TaskRunner pid=2026458) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'forward_micro_batch_size': None, (TaskRunner pid=2026458) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'grad_clip': 1.0, (TaskRunner pid=2026458) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2026458) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2026458) 'enable_activation_offload': False, (TaskRunner pid=2026458) 'enable_gradient_checkpointing': True, (TaskRunner pid=2026458) 'external_lib': None, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'entropy_checkpointing': False, (TaskRunner pid=2026458) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2026458) 'forward_only': False, (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'full_determinism': False, (TaskRunner pid=2026458) 'model_dtype': 'fp32', (TaskRunner pid=2026458) 'offload_policy': False, (TaskRunner pid=2026458) 'optimizer_offload': False, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'seed': 42, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_orig_params': False, (TaskRunner pid=2026458) 'use_torch_compile': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'lora_alpha': 16, (TaskRunner pid=2026458) 'lora_rank': 0, (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2026458) 'target_modules': 'all-linear', (TaskRunner pid=2026458) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2026458) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_remove_padding': False, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2026458) 'betas': [0.9, 0.999], (TaskRunner pid=2026458) 'clip_grad': 1.0, (TaskRunner pid=2026458) 'lr': 1e-05, (TaskRunner pid=2026458) 'lr_scheduler_type': 'constant', (TaskRunner pid=2026458) 'lr_warmup_steps': -1, (TaskRunner pid=2026458) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2026458) 'min_lr_ratio': 0.0, (TaskRunner pid=2026458) 'num_cycles': 0.5, (TaskRunner pid=2026458) 'optimizer': 'AdamW', (TaskRunner pid=2026458) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2026458) 'override_optimizer_config': None, (TaskRunner pid=2026458) 'total_training_steps': -1, (TaskRunner pid=2026458) 'warmup_style': None, (TaskRunner pid=2026458) 'weight_decay': 0.01}, (TaskRunner pid=2026458) 'ppo_epochs': 1, (TaskRunner pid=2026458) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'ppo_micro_batch_size': None, (TaskRunner pid=2026458) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'ppo_mini_batch_size': 8, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'rollout_n': 8, (TaskRunner pid=2026458) 'shuffle': False, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False}, (TaskRunner pid=2026458) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2026458) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2026458) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2026458) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2026458) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2026458) 'dataloader_num_workers': 8, (TaskRunner pid=2026458) 'filter_overlong_prompts': True, (TaskRunner pid=2026458) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2026458) 'image_key': 'images', (TaskRunner pid=2026458) 'image_patch_size': 14, (TaskRunner pid=2026458) 'max_prompt_length': 2048, (TaskRunner pid=2026458) 'max_response_length': 8192, (TaskRunner pid=2026458) 'prompt_key': 'prompt', (TaskRunner pid=2026458) 'return_full_prompt': False, (TaskRunner pid=2026458) 'return_multi_modal_inputs': True, (TaskRunner pid=2026458) 'return_raw_chat': True, (TaskRunner pid=2026458) 'return_raw_input_ids': False, (TaskRunner pid=2026458) 'reward_fn_key': 'data_source', (TaskRunner pid=2026458) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2026458) 'seed': None, (TaskRunner pid=2026458) 'shuffle': True, (TaskRunner pid=2026458) 'tokenizer': None, (TaskRunner pid=2026458) 'tool_config_path': None, (TaskRunner pid=2026458) 'train_batch_size': 32, (TaskRunner pid=2026458) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/train.parquet'], (TaskRunner pid=2026458) 'train_max_samples': 3200, (TaskRunner pid=2026458) 'truncation': 'error', (TaskRunner pid=2026458) 'trust_remote_code': True, (TaskRunner pid=2026458) 'use_shm': False, (TaskRunner pid=2026458) 'val_batch_size': None, (TaskRunner pid=2026458) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/chemistry/test.parquet'], (TaskRunner pid=2026458) 'val_max_samples': -1, (TaskRunner pid=2026458) 'validation_shuffle': False, (TaskRunner pid=2026458) 'video_key': 'videos'}, (TaskRunner pid=2026458) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2026458) 'cuda-memory-usage': 'true', (TaskRunner pid=2026458) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2026458) 'capture-range-end': None, (TaskRunner pid=2026458) 'cuda-graph-trace': 'graph', (TaskRunner pid=2026458) 'cuda-memory-usage': 'true', (TaskRunner pid=2026458) 'kill': 'none', (TaskRunner pid=2026458) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2026458) 'torch_memory': {'context': 'all', (TaskRunner pid=2026458) 'kw_args': {}, (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'stacks': 'all', (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2026458) 'profile_continuous_steps': False, (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'steps': None, (TaskRunner pid=2026458) 'tool': None}, (TaskRunner pid=2026458) 'max_model_len': 10240, (TaskRunner pid=2026458) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_chemistry_rlsd_tr_Qwen3_4B', (TaskRunner pid=2026458) 'include_dashboard': False, (TaskRunner pid=2026458) 'num_cpus': None, (TaskRunner pid=2026458) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'TASK': 'datasets/sciknoweval/chemistry', (TaskRunner pid=2026458) 'USER': 'root'}}}, (TaskRunner pid=2026458) 'timeline_json_file': None}, (TaskRunner pid=2026458) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2026458) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2026458) 'name': 'custom_reward_manager', (TaskRunner pid=2026458) 'path': None}, (TaskRunner pid=2026458) 'name': 'naive', (TaskRunner pid=2026458) 'source': 'register'}, (TaskRunner pid=2026458) 'reward_model': {'enable': False, (TaskRunner pid=2026458) 'enable_resource_pool': False, (TaskRunner pid=2026458) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2026458) 'launch_reward_fn_async': False, (TaskRunner pid=2026458) 'max_length': None, (TaskRunner pid=2026458) 'micro_batch_size': None, (TaskRunner pid=2026458) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2026458) 'model': {'external_lib': None, (TaskRunner pid=2026458) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2026458) 'forward_prefetch': False, (TaskRunner pid=2026458) 'fsdp_size': -1, (TaskRunner pid=2026458) 'param_offload': False, (TaskRunner pid=2026458) 'reshard_after_forward': True, (TaskRunner pid=2026458) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2026458) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2026458) 'override_config': {}, (TaskRunner pid=2026458) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2026458) 'trust_remote_code': False, (TaskRunner pid=2026458) 'use_fused_kernels': False, (TaskRunner pid=2026458) 'use_remove_padding': False, (TaskRunner pid=2026458) 'use_shm': False}, (TaskRunner pid=2026458) 'n_gpus_per_node': 8, (TaskRunner pid=2026458) 'nnodes': 0, (TaskRunner pid=2026458) 'num_workers': 1, (TaskRunner pid=2026458) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2026458) 'all_ranks': False, (TaskRunner pid=2026458) 'enable': False, (TaskRunner pid=2026458) 'ranks': [], (TaskRunner pid=2026458) 'save_path': 'outputs/profile', (TaskRunner pid=2026458) 'tool': None, (TaskRunner pid=2026458) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2026458) 'analysis': True, (TaskRunner pid=2026458) 'contents': [], (TaskRunner pid=2026458) 'discrete': False, (TaskRunner pid=2026458) 'level': 'level0'}, (TaskRunner pid=2026458) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2026458) 'discrete': False}, (TaskRunner pid=2026458) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2026458) 'step_end': None, (TaskRunner pid=2026458) 'step_start': 0}, (TaskRunner pid=2026458) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2026458) 'stack_depth': 32, (TaskRunner pid=2026458) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2026458) 'reward_loop_class_name': None, (TaskRunner pid=2026458) 'reward_loop_module_path': None, (TaskRunner pid=2026458) 'reward_loop_source': 'register', (TaskRunner pid=2026458) 'reward_manager': 'naive', (TaskRunner pid=2026458) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2026458) 'cudagraph_capture_sizes': None, (TaskRunner pid=2026458) 'data_parallel_size': 1, (TaskRunner pid=2026458) 'disable_log_stats': True, (TaskRunner pid=2026458) 'dtype': 'bfloat16', (TaskRunner pid=2026458) 'enable_chunked_prefill': True, (TaskRunner pid=2026458) 'enable_prefix_caching': True, (TaskRunner pid=2026458) 'enforce_eager': True, (TaskRunner pid=2026458) 'engine_kwargs': {}, (TaskRunner pid=2026458) 'expert_parallel_size': 1, (TaskRunner pid=2026458) 'free_cache_engine': True, (TaskRunner pid=2026458) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2026458) 'limit_images': None, (TaskRunner pid=2026458) 'load_format': 'auto', (TaskRunner pid=2026458) 'max_model_len': None, (TaskRunner pid=2026458) 'max_num_batched_tokens': 8192, (TaskRunner pid=2026458) 'max_num_seqs': 1024, (TaskRunner pid=2026458) 'name': '???', (TaskRunner pid=2026458) 'prompt_length': 2048, (TaskRunner pid=2026458) 'response_length': 2048, (TaskRunner pid=2026458) 'skip_tokenizer_init': False, (TaskRunner pid=2026458) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2026458) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2026458) 'memory_limit_mb': 1024, (TaskRunner pid=2026458) 'url': None}, (TaskRunner pid=2026458) 'strategy': 'fsdp', (TaskRunner pid=2026458) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2026458) 'use_dynamic_bsz': False, (TaskRunner pid=2026458) 'use_reward_loop': False}, (TaskRunner pid=2026458) 'trainer': {'balance_batch': True, (TaskRunner pid=2026458) 'critic_warmup': 0, (TaskRunner pid=2026458) 'default_hdfs_dir': None, (TaskRunner pid=2026458) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'del_local_ckpt_after_load': False, (TaskRunner pid=2026458) 'device': 'cuda', (TaskRunner pid=2026458) 'esi_redundant_time': 0, (TaskRunner pid=2026458) 'experiment_name': 'qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2026458) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2026458) 'log_val_generations': 0, (TaskRunner pid=2026458) 'logger': ['console', 'wandb'], (TaskRunner pid=2026458) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2026458) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2026458) 'n_gpus_per_node': 8, (TaskRunner pid=2026458) 'nnodes': 1, (TaskRunner pid=2026458) 'project_name': 'SDPO-root', (TaskRunner pid=2026458) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2026458) 'resume_from_path': None, (TaskRunner pid=2026458) 'resume_mode': 'auto', (TaskRunner pid=2026458) 'rollout_data_dir': None, (TaskRunner pid=2026458) 'save_freq': 10, (TaskRunner pid=2026458) 'test_freq': 10, (TaskRunner pid=2026458) 'total_epochs': 30, (TaskRunner pid=2026458) 'total_training_steps': 100, (TaskRunner pid=2026458) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2026458) 'val_before_train': False, (TaskRunner pid=2026458) 'val_only': False, (TaskRunner pid=2026458) 'validation_data_dir': None}, (TaskRunner pid=2026458) 'transfer_queue': {'enable': False}, (TaskRunner pid=2026458) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/chemistry', (TaskRunner pid=2026458) 'dir': '/users/root/SDPO', (TaskRunner pid=2026458) 'log_dir': '/users/root/output', (TaskRunner pid=2026458) 'task': 'datasets/sciknoweval/chemistry'}} (TaskRunner pid=2026458) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2026458) use_critic=need_critic(config), (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2026458) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2026458) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: eb9682da-997b-4a49-aefd-e3ec8c865f3b)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:07,289:'(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1010)')))"), '(Request ID: eb9682da-997b-4a49-aefd-e3ec8c865f3b)')' thrown while requesting HEAD https://huggingface.co/Qwen/Qwen3-4B/resolve/main/tokenizer_config.json (TaskRunner pid=2026458) Retrying in 1s [Retry 1/5]. (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:07,290:Retrying in 1s [Retry 1/5]. (TaskRunner pid=2026458) Using dataset class: RLHFDataset (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2026458) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2026458) dataset len: 1890 (TaskRunner pid=2026458) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2026458) WARNING:2026-07-02 06:19:12,124:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/1890 [00:00 (TaskRunner pid=2026458) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2026458) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2026458) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 210/210 [00:00<00:00, 301.55 examples/s] (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2026458) self.use_critic = need_critic(self.config) (WorkerDict pid=2026872) [W702 06:19:21.430329774 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:52047 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2026871) [Gloo] Rank 3 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2026873) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2026873) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2026869) [W702 06:19:22.501122826 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:52047 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2026868) Model config after override: Qwen3Config { (WorkerDict pid=2026868) "architectures": [ (WorkerDict pid=2026868) "Qwen3ForCausalLM" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "attention_bias": false, (WorkerDict pid=2026868) "attention_dropout": 0.0, (WorkerDict pid=2026868) "dtype": "bfloat16", (WorkerDict pid=2026868) "eos_token_id": 151645, (WorkerDict pid=2026868) "head_dim": 128, (WorkerDict pid=2026868) "hidden_act": "silu", (WorkerDict pid=2026868) "hidden_size": 2560, (WorkerDict pid=2026868) "initializer_range": 0.02, (WorkerDict pid=2026868) "intermediate_size": 9728, (WorkerDict pid=2026868) "layer_types": [ (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "max_position_embeddings": 40960, (WorkerDict pid=2026868) "max_window_layers": 36, (WorkerDict pid=2026868) "model_type": "qwen3", (WorkerDict pid=2026868) "num_attention_heads": 32, (WorkerDict pid=2026868) "num_hidden_layers": 36, (WorkerDict pid=2026868) "num_key_value_heads": 8, (WorkerDict pid=2026868) "pad_token_id": 151643, (WorkerDict pid=2026868) "rms_norm_eps": 1e-06, (WorkerDict pid=2026868) "rope_scaling": null, (WorkerDict pid=2026868) "rope_theta": 1000000, (WorkerDict pid=2026868) "sliding_window": null, (WorkerDict pid=2026868) "tie_word_embeddings": true, (WorkerDict pid=2026868) "transformers_version": "4.57.1", (WorkerDict pid=2026868) "use_cache": true, (WorkerDict pid=2026868) "use_sliding_window": false, (WorkerDict pid=2026868) "vocab_size": 151936 (WorkerDict pid=2026868) } (WorkerDict pid=2026868) (WorkerDict pid=2026873) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2026873) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2026873) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2026873) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2026868) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2026871) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2026870) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.94s/it] [repeated 12x across cluster] (WorkerDict pid=2026870) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 4x across cluster] (WorkerDict pid=2026868) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 3.00s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.01s/it] [repeated 2x across cluster] (WorkerDict pid=2026868) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2026868) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2026868) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2026868) Actor use_remove_padding=True (WorkerDict pid=2026868) Actor use_fused_kernels=False (WorkerDict pid=2026868) Actor use_prefix_grouper=False (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2026871) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2026871) [Gloo] Rank 1 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2026871) [Gloo] Rank 1 is connected to 1 peer ranks. Expected number of connected peer ranks is : 1 (WorkerDict pid=2026871) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2026871) warnings.warn( (WorkerDict pid=2026869) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2026869) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2026868) reference model: Qwen/Qwen3-4B (WorkerDict pid=2026871) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2026871) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2026868) Model config after override: Qwen3Config { (WorkerDict pid=2026868) "architectures": [ (WorkerDict pid=2026868) "Qwen3ForCausalLM" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "attention_bias": false, (WorkerDict pid=2026868) "attention_dropout": 0.0, (WorkerDict pid=2026868) "dtype": "bfloat16", (WorkerDict pid=2026868) "eos_token_id": 151645, (WorkerDict pid=2026868) "head_dim": 128, (WorkerDict pid=2026868) "hidden_act": "silu", (WorkerDict pid=2026868) "hidden_size": 2560, (WorkerDict pid=2026868) "initializer_range": 0.02, (WorkerDict pid=2026868) "intermediate_size": 9728, (WorkerDict pid=2026868) "layer_types": [ (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention", (WorkerDict pid=2026868) "full_attention" (WorkerDict pid=2026868) ], (WorkerDict pid=2026868) "max_position_embeddings": 40960, (WorkerDict pid=2026868) "max_window_layers": 36, (WorkerDict pid=2026868) "model_type": "qwen3", (WorkerDict pid=2026868) "num_attention_heads": 32, (WorkerDict pid=2026868) "num_hidden_layers": 36, (WorkerDict pid=2026868) "num_key_value_heads": 8, (WorkerDict pid=2026868) "pad_token_id": 151643, (WorkerDict pid=2026868) "rms_norm_eps": 1e-06, (WorkerDict pid=2026868) "rope_scaling": null, (WorkerDict pid=2026868) "rope_theta": 1000000, (WorkerDict pid=2026868) "sliding_window": null, (WorkerDict pid=2026868) "tie_word_embeddings": true, (WorkerDict pid=2026868) "transformers_version": "4.57.1", (WorkerDict pid=2026868) "use_cache": true, (WorkerDict pid=2026868) "use_sliding_window": false, (WorkerDict pid=2026868) "vocab_size": 151936 (WorkerDict pid=2026868) } (WorkerDict pid=2026868) (WorkerDict pid=2026871) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2026868) Ref use_remove_padding=True (WorkerDict pid=2026868) Ref use_fused_kernels=False (WorkerDict pid=2026868) Ref use_prefix_grouper=False (TaskRunner pid=2026458) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2026458) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2026874) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.90s/it] [repeated 12x across cluster] (WorkerDict pid=2026873) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.89s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 2x across cluster] (WorkerDict pid=2026874) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 4x across cluster] (vLLMHttpServer pid=2027750) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2027750) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2027750) ['serve', (vLLMHttpServer pid=2027750) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2027750) '--dtype', (vLLMHttpServer pid=2027750) 'bfloat16', (vLLMHttpServer pid=2027750) '--load_format', (vLLMHttpServer pid=2027750) 'dummy', (vLLMHttpServer pid=2027750) '--trust_remote_code', (vLLMHttpServer pid=2027750) '--max_model_len', (vLLMHttpServer pid=2027750) '40960', (vLLMHttpServer pid=2027750) '--max_num_seqs', (vLLMHttpServer pid=2027750) '1024', (vLLMHttpServer pid=2027750) '--enable_chunked_prefill', (vLLMHttpServer pid=2027750) '--max_num_batched_tokens', (vLLMHttpServer pid=2027750) '10240', (vLLMHttpServer pid=2027750) '--enable_prefix_caching', (vLLMHttpServer pid=2027750) '--enable_sleep_mode', (vLLMHttpServer pid=2027750) '--logprobs_mode', (vLLMHttpServer pid=2027750) 'processed_logprobs', (vLLMHttpServer pid=2027750) '--disable_custom_all_reduce', (vLLMHttpServer pid=2027750) '--gpu_memory_utilization', (vLLMHttpServer pid=2027750) '0.8', (vLLMHttpServer pid=2027750) '--disable_log_stats', (vLLMHttpServer pid=2027750) '--tensor_parallel_size', (vLLMHttpServer pid=2027750) '2', (vLLMHttpServer pid=2027750) '--seed', (vLLMHttpServer pid=2027750) '0', (vLLMHttpServer pid=2027750) '--override_generation_config', (vLLMHttpServer pid=2027750) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2027750) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2027750) '--hf_overrides', (vLLMHttpServer pid=2027750) '{}', (vLLMHttpServer pid=2027750) '--scheduling_policy', (vLLMHttpServer pid=2027750) 'fcfs'] (WorkerDict pid=2026872) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2026872) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (vLLMHttpServer pid=2027750) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2027750) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2027750) WARNING 07-02 06:20:18 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2026868) WARNING 07-02 06:20:26 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2027751) WARNING 07-02 06:20:19 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2026872) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2026868) 2026-07-02 06:20:40,724 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2026868) 2026-07-02 06:20:40,743 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (vLLMHttpServer pid=2027753) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2027753) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2027753) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2027753) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2026868) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which molecule is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a complex structure with a benzene ring substituted with a chlorine atom and a trifluoromethyl group (C(F)(F)F). The key part of the product is the "Cc2ccc(Cl)cc2C(F)(F)F" fragment, which suggests that the reactant must contain a similar structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A contains a ketone group (O=C) and a nitrogen-containing ring, but it does not match the specific substituents in the product. (TaskRunner pid=2026458) - Option B contains a chlorine atom and a nitrogen, but it does not include the trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) - Option C contains a trifluoromethyl group (C(F)(F)F) and a chlorine atom, and it includes a structure that matches the "Cc2ccc(Cl)cc2" part of the product. (TaskRunner pid=2026458) - Option D contains a trifluoromethyl group but lacks the chlorine atom in the correct position. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C is the only one that includes both the chlorine and the trifluoromethyl group in the correct positions, matching the structure of the product. Therefore, it is the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) C (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 0.0 (TaskRunner pid=2026458) [acc] 0.0 (TaskRunner pid=2026458) [pred] C (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2026458) step:10 - global_seqlen/min:17490 - global_seqlen/max:20901 - global_seqlen/minmax_diff:3411 - global_seqlen/balanced_min:19357 - global_seqlen/balanced_max:19367 - global_seqlen/mean:19364.75 - actor/entropy:0.24537719786167145 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744035601615906 - training/rollout_probs_diff_mean:0.005785205401480198 - training/rollout_probs_diff_std:0.014700246043503284 - training/rollout_actor_probs_pearson_corr:0.9979208707809448 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9998391270637512 - rollout_corr/rollout_is_ratio_fraction_high:1.1633588655968197e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.980152829783037e-05 - rollout_corr/rollout_is_max:2.010491132736206 - rollout_corr/rollout_is_min:0.23784410953521729 - rollout_corr/rollout_is_std:0.04395073652267456 - rollout_corr/rollout_is_eff_sample_size:0.9980714630669532 - rollout_corr/rollout_is_seq_mean:0.9998910427093506 - rollout_corr/rollout_is_seq_std:0.0027966259513050318 - rollout_corr/rollout_is_seq_max:1.0104008913040161 - rollout_corr/rollout_is_seq_min:0.9919784069061279 - rollout_corr/rollout_is_seq_max_deviation:0.010400891304016113 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3026330736465752) - rollout_corr/training_log_ppl:np.float64(0.26078601760673337) - rollout_corr/kl:np.float64(0.0013049291076470126) - rollout_corr/k3_kl:np.float64(0.0011964667813799679) - rollout_corr/rollout_ppl:np.float64(1.3009595507755876) - rollout_corr/rollout_log_ppl:np.float64(0.25948108817101456) - rollout_corr/log_ppl_diff:np.float64(0.0013049294357188046) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024894755915738642) - rollout_corr/log_ppl_diff_max:np.float64(0.017253637313842773) - rollout_corr/log_ppl_diff_min:np.float64(-0.008967161178588867) - rollout_corr/ppl_ratio:np.float64(1.0013103527016938) - rollout_corr/chi2_token:np.float64(0.002327824244275689) - rollout_corr/chi2_seq:np.float64(0.852935308823362) - actor/pg_loss:np.float64(0.00642693554982543) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010866172510759498) - actor/ppo_kl:np.float64(0.00015771930741692586) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(177288.61759452336) - self_distillation/token_reweight_w_std:np.float64(2583470.6754472964) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025011864490807) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1041747338604182) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.5560944229364395) - perf/mfu/actor:np.float64(0.03574857500362935) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(141.2105484008789) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/reward/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.16058361739063698) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7451095238095238) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04477536267518606) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.46405714285714283) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11915383713937476) - val-aux/sciknoweval/score/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/score/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/score/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/score/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/score/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.16058361739063698) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7451095238095238) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04477536267518606) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.46405714285714283) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11915383713937476) - val-core/sciknoweval/acc/mean@16:np.float64(0.4398809523809524) - val-aux/sciknoweval/acc/std@16:np.float64(0.2550574423136475) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.5466857142857143) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.22323689062871058) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.3330666666666666) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.20976723560246902) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.44009047619047625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.25489768792129186) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6363809523809525) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.16780774930381429) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.2502714285714286) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.14555290883567562) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.4538952380952381) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.209952783081799) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7015523809523809) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.1130187029836996) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.1943) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.087999239033555) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.4615809523809523) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.16058361739063698) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7451095238095238) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06576006624725168) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.16135714285714284) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04477536267518606) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.46405714285714283) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11915383713937476) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994571428571428) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015133869749850571) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997476190476191) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0010668260935279934) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9989428571428571) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0019790054535129544) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999190476190476) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0006155763481184213) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9981333333333333) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023247461032216924) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9970238095238095) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0023053472298853674) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001270969514735043 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001270969514735043 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:335.7734375 - response_length/max:1122.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:335.7734375 - response_length_non_aborted/max:1122.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.375 - prompt_length/max:701.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.7655776143074036e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4987823646515608) - timing_s/agent_loop/generate_sequences/max:np.float64(7.95271722227335) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.580353087221738) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.95271722227335) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:1122 - timing_s/gen:13.539362616837025 - timing_s/reward:0.05532005988061428 - timing_s/old_log_prob:2.3085584342479706 - timing_s/adv:0.48644337616860867 - timing_s/update_actor:15.44553491845727 - timing_s/step:31.924703411757946 - timing_s/testing:117.72085026092827 - timing_s/save_checkpoint:6.645242588594556 - timing_s/stop_profile:0.0001191999763250351 - timing_per_token_ms/adv:0.0031400055265921887 - timing_per_token_ms/gen:0.1575113731919894 - timing_per_token_ms/update_actor:0.09970135761149299 - perf/total_num_tokens:154918 - perf/time_per_step:31.924703411757946 - perf/throughput:606.5757213226894 (TaskRunner pid=2026458) Training Progress: 10%|█ | 10/100 [08:48<1:56:19, 77.55s/it] (TaskRunner pid=2026458) step:11 - global_seqlen/min:17717 - global_seqlen/max:22105 - global_seqlen/minmax_diff:4388 - global_seqlen/balanced_min:20139 - global_seqlen/balanced_max:20427 - global_seqlen/mean:20176.875 - actor/entropy:0.28003615140914917 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2986578345298767 - training/rollout_probs_diff_mean:0.006229304242879152 - training/rollout_probs_diff_std:0.014922214671969414 - training/rollout_actor_probs_pearson_corr:0.9980378746986389 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9996895790100098 - rollout_corr/rollout_is_ratio_fraction_high:2.3571841666125692e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.071552681736648e-05 - rollout_corr/rollout_is_max:2.091331720352173 - rollout_corr/rollout_is_min:0.056948471814394 - rollout_corr/rollout_is_std:0.04615355655550957 - rollout_corr/rollout_is_eff_sample_size:0.9978729526678736 - rollout_corr/rollout_is_seq_mean:0.9996609091758728 - rollout_corr/rollout_is_seq_std:0.002769676735624671 - rollout_corr/rollout_is_seq_max:1.0097713470458984 - rollout_corr/rollout_is_seq_min:0.9914740920066833 - rollout_corr/rollout_is_seq_max_deviation:0.009771347045898438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3445568550378084) - rollout_corr/training_log_ppl:np.float64(0.2928216397704091) - rollout_corr/kl:np.float64(0.0013540790551616055) - rollout_corr/k3_kl:np.float64(0.001210083299042708) - rollout_corr/rollout_ppl:np.float64(1.3426842936314642) - rollout_corr/rollout_log_ppl:np.float64(0.2914675589709077) - rollout_corr/log_ppl_diff:np.float64(0.0013540807995013893) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025300010456703603) - rollout_corr/log_ppl_diff_max:np.float64(0.011778771877288818) - rollout_corr/log_ppl_diff_min:np.float64(-0.006693422794342041) - rollout_corr/ppl_ratio:np.float64(1.001359449699521) - rollout_corr/chi2_token:np.float64(0.0021713071037083864) - rollout_corr/chi2_seq:np.float64(0.5860715808812529) - actor/pg_loss:np.float64(0.009074105182662606) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010309692956980143) - actor/ppo_kl:np.float64(-0.00014117327255291912) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(239610.312335568) - self_distillation/token_reweight_w_std:np.float64(3316290.997419097) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019973907619715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14387543653720059) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.6877006888389587) - perf/mfu/actor:np.float64(0.037142691887104826) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9859390258789) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006922754924744368 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006922754924744368 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:331.43359375 - response_length/max:1206.0 - response_length/min:142.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:331.43359375 - response_length_non_aborted/max:1206.0 - response_length_non_aborted/min:142.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.09375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012538395822048187 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7413379289209843) - timing_s/agent_loop/generate_sequences/max:np.float64(8.170563256368041) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.67358700458135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.170563256368041) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:1206 - timing_s/gen:14.099589532241225 - timing_s/reward:0.05785256810486317 - timing_s/old_log_prob:2.514467492699623 - timing_s/adv:0.5351101234555244 - timing_s/update_actor:15.359056254848838 - timing_s/step:32.6669915933162 - timing_s/stop_profile:0.0001175031065940857 - timing_per_token_ms/adv:0.003315120177527023 - timing_per_token_ms/gen:0.16617664186407563 - timing_per_token_ms/update_actor:0.09515259582349124 - perf/total_num_tokens:161415 - perf/time_per_step:32.6669915933162 - perf/throughput:617.6532951423746 (TaskRunner pid=2026458) Training Progress: 11%|█ | 11/100 [09:20<1:34:40, 63.83s/it] (TaskRunner pid=2026458) step:12 - global_seqlen/min:17937 - global_seqlen/max:22358 - global_seqlen/minmax_diff:4421 - global_seqlen/balanced_min:19749 - global_seqlen/balanced_max:19763 - global_seqlen/mean:19757.25 - actor/entropy:0.2599850594997406 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29290565848350525 - training/rollout_probs_diff_mean:0.00601433590054512 - training/rollout_probs_diff_std:0.014930167235434055 - training/rollout_actor_probs_pearson_corr:0.9979494214057922 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.4609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.999984085559845 - rollout_corr/rollout_is_ratio_fraction_high:2.223754199803807e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.33563111780677e-05 - rollout_corr/rollout_is_max:2.084258556365967 - rollout_corr/rollout_is_min:0.48210135102272034 - rollout_corr/rollout_is_std:0.04539177566766739 - rollout_corr/rollout_is_eff_sample_size:0.9979434670549656 - rollout_corr/rollout_is_seq_mean:0.9999362230300903 - rollout_corr/rollout_is_seq_std:0.002637566300109029 - rollout_corr/rollout_is_seq_max:1.0081006288528442 - rollout_corr/rollout_is_seq_min:0.9917522072792053 - rollout_corr/rollout_is_seq_max_deviation:0.008247792720794678 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3254084084182978) - rollout_corr/training_log_ppl:np.float64(0.2779389991774224) - rollout_corr/kl:np.float64(0.0013704395118168122) - rollout_corr/k3_kl:np.float64(0.0011261843454803966) - rollout_corr/rollout_ppl:np.float64(1.3235342632979155) - rollout_corr/rollout_log_ppl:np.float64(0.27656855969689786) - rollout_corr/log_ppl_diff:np.float64(0.0013704394805245101) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024474224192090333) - rollout_corr/log_ppl_diff_max:np.float64(0.011240929365158081) - rollout_corr/log_ppl_diff_min:np.float64(-0.010027527809143066) - rollout_corr/ppl_ratio:np.float64(1.001375619089231) - rollout_corr/chi2_token:np.float64(0.001807245658710599) - rollout_corr/chi2_seq:np.float64(1.2864351340103894) - actor/pg_loss:np.float64(0.0068493077997118235) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010652919690983254) - actor/ppo_kl:np.float64(0.00020145679512317827) - actor/pg_clipfrac_lower:np.float64(1.669337689236272e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4609375) - self_distillation/token_reweight_w_mean:np.float64(145999.17218339257) - self_distillation/token_reweight_w_std:np.float64(2374372.460695313) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004515508189797) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1246801737579517) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.5914773046970367) - perf/mfu/actor:np.float64(0.03625096042724267) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.91592407226562) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.4609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004239031113684177 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004239031113684177 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:351.3203125 - response_length/max:1148.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:351.3203125 - response_length_non_aborted/max:1148.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.09375 - prompt_length/max:665.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.013755116611719131 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6791318338364363) - timing_s/agent_loop/generate_sequences/max:np.float64(8.12350033968687) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.846359725270304) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.12350033968687) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:264 - timing_s/agent_loop/slowest/response_length:1148 - timing_s/gen:13.899831658229232 - timing_s/reward:0.05922466889023781 - timing_s/old_log_prob:2.4094871655106544 - timing_s/adv:0.5697984360158443 - timing_s/update_actor:15.474863145500422 - timing_s/step:32.48048596084118 - timing_s/stop_profile:0.0001192968338727951 - timing_per_token_ms/adv:0.003604995862378648 - timing_per_token_ms/gen:0.15454904109752532 - timing_per_token_ms/update_actor:0.09790623154475206 - perf/total_num_tokens:158058 - perf/time_per_step:32.48048596084118 - perf/throughput:608.2806157463147 (TaskRunner pid=2026458) Training Progress: 12%|█▏ | 12/100 [09:53<1:19:39, 54.31s/it] (TaskRunner pid=2026458) step:13 - global_seqlen/min:17460 - global_seqlen/max:22387 - global_seqlen/minmax_diff:4927 - global_seqlen/balanced_min:19538 - global_seqlen/balanced_max:19656 - global_seqlen/mean:19562.0 - actor/entropy:0.28330478072166443 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2932348847389221 - training/rollout_probs_diff_mean:0.006090556271374226 - training/rollout_probs_diff_std:0.014429626986384392 - training/rollout_actor_probs_pearson_corr:0.998181939125061 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.68359375 - self_distillation/correct_sample_fraction:0.47265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.68359375 - rollout_corr/rollout_is_mean:0.9998684525489807 - rollout_corr/rollout_is_ratio_fraction_high:2.3373224394163117e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.3373224394163117e-05 - rollout_corr/rollout_is_max:2.1884407997131348 - rollout_corr/rollout_is_min:0.4281519055366516 - rollout_corr/rollout_is_std:0.04431807994842529 - rollout_corr/rollout_is_eff_sample_size:0.9980391641302073 - rollout_corr/rollout_is_seq_mean:0.9998126029968262 - rollout_corr/rollout_is_seq_std:0.0026776245795190334 - rollout_corr/rollout_is_seq_max:1.009365439414978 - rollout_corr/rollout_is_seq_min:0.9913782477378845 - rollout_corr/rollout_is_seq_max_deviation:0.009365439414978027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3591533433645964) - rollout_corr/training_log_ppl:np.float64(0.30308022926328704) - rollout_corr/kl:np.float64(0.0012214116412607723) - rollout_corr/k3_kl:np.float64(0.0011643693818541578) - rollout_corr/rollout_ppl:np.float64(1.3574783634394407) - rollout_corr/rollout_log_ppl:np.float64(0.3018588193517644) - rollout_corr/log_ppl_diff:np.float64(0.0012214099115226418) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024031201901379973) - rollout_corr/log_ppl_diff_max:np.float64(0.011142253875732422) - rollout_corr/log_ppl_diff_min:np.float64(-0.00809323787689209) - rollout_corr/ppl_ratio:np.float64(1.0012265297118574) - rollout_corr/chi2_token:np.float64(0.0022032533306628466) - rollout_corr/chi2_seq:np.float64(1.1880777331534773) - actor/pg_loss:np.float64(0.008368437876924872) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011567205356186605) - actor/ppo_kl:np.float64(-1.6140308314760432e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.68359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.47265625) - self_distillation/token_reweight_w_mean:np.float64(93498.56042674859) - self_distillation/token_reweight_w_std:np.float64(1510537.690864286) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999996354104951) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12338341507711448) - self_distillation/empty_target_batch:np.float64(0.31640625) - actor/grad_norm:np.float64(0.620448037981987) - perf/mfu/actor:np.float64(0.036300476348807696) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.69003295898438) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.47265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.47265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003617006354033947 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003617006354033947 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:334.25 - response_length/max:1159.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:334.25 - response_length_non_aborted/max:1159.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:612.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014793872833251953 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3433514721691608) - timing_s/agent_loop/generate_sequences/max:np.float64(7.973012773320079) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.6053173123218585) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.973012773320079) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:1159 - timing_s/gen:13.718764901161194 - timing_s/reward:0.05917247012257576 - timing_s/old_log_prob:2.3764582853764296 - timing_s/adv:0.49192045256495476 - timing_s/update_actor:15.186074122786522 - timing_s/step:31.921324100345373 - timing_s/stop_profile:0.00019265897572040558 - timing_per_token_ms/adv:0.003143342018741404 - timing_per_token_ms/gen:0.1603258800154403 - timing_per_token_ms/update_actor:0.09703809760496448 - perf/total_num_tokens:156496 - perf/time_per_step:31.921324100345373 - perf/throughput:612.8191906609648 (TaskRunner pid=2026458) Training Progress: 13%|█▎ | 13/100 [10:25<1:08:56, 47.54s/it] (TaskRunner pid=2026458) step:14 - global_seqlen/min:16941 - global_seqlen/max:22560 - global_seqlen/minmax_diff:5619 - global_seqlen/balanced_min:18841 - global_seqlen/balanced_max:18844 - global_seqlen/mean:18842.75 - actor/entropy:0.2922348976135254 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5466010570526123 - training/rollout_probs_diff_mean:0.006264000199735165 - training/rollout_probs_diff_std:0.014866359531879425 - training/rollout_actor_probs_pearson_corr:0.9981194138526917 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0002233982086182 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:7.292350346688181e-05 - rollout_corr/rollout_is_max:1.9629987478256226 - rollout_corr/rollout_is_min:0.12186871469020844 - rollout_corr/rollout_is_std:0.04516667500138283 - rollout_corr/rollout_is_eff_sample_size:0.9979653119171001 - rollout_corr/rollout_is_seq_mean:1.0002156496047974 - rollout_corr/rollout_is_seq_std:0.00266577722504735 - rollout_corr/rollout_is_seq_max:1.0086764097213745 - rollout_corr/rollout_is_seq_min:0.9924325942993164 - rollout_corr/rollout_is_seq_max_deviation:0.008676409721374512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.356894207186997) - rollout_corr/training_log_ppl:np.float64(0.30215946782846004) - rollout_corr/kl:np.float64(0.0010239662146345752) - rollout_corr/k3_kl:np.float64(0.0012049657070747344) - rollout_corr/rollout_ppl:np.float64(1.3555219103582203) - rollout_corr/rollout_log_ppl:np.float64(0.3011355001362972) - rollout_corr/log_ppl_diff:np.float64(0.0010239676921628416) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023661727900616825) - rollout_corr/log_ppl_diff_max:np.float64(0.017396926879882812) - rollout_corr/log_ppl_diff_min:np.float64(-0.007369279861450195) - rollout_corr/ppl_ratio:np.float64(1.0010292823426425) - rollout_corr/chi2_token:np.float64(0.0027081489097326994) - rollout_corr/chi2_seq:np.float64(1.2070772459264845) - actor/pg_loss:np.float64(0.00797818333376199) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010150298585358541) - actor/ppo_kl:np.float64(0.00015036190526984683) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.453125) - self_distillation/token_reweight_w_mean:np.float64(119982.02733938815) - self_distillation/token_reweight_w_std:np.float64(1933813.0637409594) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014771707355976) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1388165877724532) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.6455883979797363) - perf/mfu/actor:np.float64(0.034641753809947065) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.4944953918457) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0026480345986783504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0026480345986783504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:321.3984375 - response_length/max:682.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:321.3984375 - response_length_non_aborted/max:682.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.4375 - prompt_length/max:666.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012210384011268616 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7070676069706678) - timing_s/agent_loop/generate_sequences/max:np.float64(5.696014670655131) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4708188238437288) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.696014670655131) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:413 - timing_s/agent_loop/slowest/response_length:682 - timing_s/gen:11.448512280359864 - timing_s/reward:0.05713308043777943 - timing_s/old_log_prob:2.4212917368859053 - timing_s/adv:0.4922778233885765 - timing_s/update_actor:15.44989874586463 - timing_s/step:29.956744581460953 - timing_s/stop_profile:0.00011506862938404083 - timing_per_token_ms/adv:0.003265697837288722 - timing_per_token_ms/gen:0.1391442704047238 - timing_per_token_ms/update_actor:0.10249232958209809 - perf/total_num_tokens:150742 - perf/time_per_step:29.956744581460953 - perf/throughput:628.9985865707529 (TaskRunner pid=2026458) Training Progress: 14%|█▍ | 14/100 [10:55<1:00:33, 42.25s/it] (TaskRunner pid=2026458) step:15 - global_seqlen/min:17798 - global_seqlen/max:24465 - global_seqlen/minmax_diff:6667 - global_seqlen/balanced_min:20570 - global_seqlen/balanced_max:20574 - global_seqlen/mean:20572.75 - actor/entropy:0.28748324513435364 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564783573150635 - training/rollout_probs_diff_mean:0.005869686137884855 - training/rollout_probs_diff_std:0.014366100542247295 - training/rollout_actor_probs_pearson_corr:0.9982949495315552 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.4921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0001139640808105 - rollout_corr/rollout_is_ratio_fraction_high:5.4864267440279946e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3891413952223957e-05 - rollout_corr/rollout_is_max:3.0497446060180664 - rollout_corr/rollout_is_min:0.38640856742858887 - rollout_corr/rollout_is_std:0.045057009905576706 - rollout_corr/rollout_is_eff_sample_size:0.9979744538008674 - rollout_corr/rollout_is_seq_mean:1.0000189542770386 - rollout_corr/rollout_is_seq_std:0.002783029805868864 - rollout_corr/rollout_is_seq_max:1.0093011856079102 - rollout_corr/rollout_is_seq_min:0.9919822812080383 - rollout_corr/rollout_is_seq_max_deviation:0.009301185607910156 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3641661452129483) - rollout_corr/training_log_ppl:np.float64(0.3061258476227522) - rollout_corr/kl:np.float64(0.0008984394618103408) - rollout_corr/k3_kl:np.float64(0.0010934443635051139) - rollout_corr/rollout_ppl:np.float64(1.3629517373628914) - rollout_corr/rollout_log_ppl:np.float64(0.3052274066139944) - rollout_corr/log_ppl_diff:np.float64(0.0008984410087577999) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023149027838371694) - rollout_corr/log_ppl_diff_max:np.float64(0.015007197856903076) - rollout_corr/log_ppl_diff_min:np.float64(-0.0058860182762146) - rollout_corr/ppl_ratio:np.float64(1.00090327905491) - rollout_corr/chi2_token:np.float64(0.0026275489944964647) - rollout_corr/chi2_seq:np.float64(1.554575996240601) - actor/pg_loss:np.float64(0.006172901834361255) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010921142766164849) - actor/ppo_kl:np.float64(-0.00014767027690254508) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4921875) - self_distillation/token_reweight_w_mean:np.float64(168825.68510642461) - self_distillation/token_reweight_w_std:np.float64(2411026.6848952193) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999142882879823) - self_distillation/token_reweight_w_clip_frac:np.float64(0.13169058857602067) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.5341654643416405) - perf/mfu/actor:np.float64(0.03786336781112096) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.43387985229492) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.4921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036114403046667576 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036114403046667576 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:355.9921875 - response_length/max:841.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:355.9921875 - response_length_non_aborted/max:841.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.90625 - prompt_length/max:494.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000149507075548172 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8174101784825325) - timing_s/agent_loop/generate_sequences/max:np.float64(6.822324391454458) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.8521247571407002) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.822324391454458) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:187 - timing_s/agent_loop/slowest/response_length:841 - timing_s/gen:12.53348901681602 - timing_s/reward:0.06065330095589161 - timing_s/old_log_prob:2.2750260774046183 - timing_s/adv:0.5170386601239443 - timing_s/update_actor:15.529155055060983 - timing_s/step:31.002764070406556 - timing_s/stop_profile:0.00011553429067134857 - timing_per_token_ms/adv:0.003141526170079014 - timing_per_token_ms/gen:0.13752813458002525 - timing_per_token_ms/update_actor:0.09435512422416172 - perf/total_num_tokens:164582 - perf/time_per_step:31.002764070406556 - perf/throughput:663.5779298026383 (TaskRunner pid=2026458) Training Progress: 15%|█▌ | 15/100 [11:26<55:04, 38.87s/it] (TaskRunner pid=2026458) step:16 - global_seqlen/min:17436 - global_seqlen/max:21462 - global_seqlen/minmax_diff:4026 - global_seqlen/balanced_min:19472 - global_seqlen/balanced_max:19474 - global_seqlen/mean:19473.375 - actor/entropy:0.29451751708984375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34674251079559326 - training/rollout_probs_diff_mean:0.006398364901542664 - training/rollout_probs_diff_std:0.014944251626729965 - training/rollout_actor_probs_pearson_corr:0.9980317950248718 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.64453125 - self_distillation/correct_sample_fraction:0.42578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.64453125 - rollout_corr/rollout_is_mean:1.000139832496643 - rollout_corr/rollout_is_ratio_fraction_high:3.57470526068937e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001429882104275748 - rollout_corr/rollout_is_max:2.075308322906494 - rollout_corr/rollout_is_min:0.33335795998573303 - rollout_corr/rollout_is_std:0.04721960052847862 - rollout_corr/rollout_is_eff_sample_size:0.9977759819851924 - rollout_corr/rollout_is_seq_mean:1.0001648664474487 - rollout_corr/rollout_is_seq_std:0.002871463308110833 - rollout_corr/rollout_is_seq_max:1.0102380514144897 - rollout_corr/rollout_is_seq_min:0.9917946457862854 - rollout_corr/rollout_is_seq_max_deviation:0.010238051414489746 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36143590323627) - rollout_corr/training_log_ppl:np.float64(0.3041792264266405) - rollout_corr/kl:np.float64(0.0010337699030245062) - rollout_corr/k3_kl:np.float64(0.0012385786424147227) - rollout_corr/rollout_ppl:np.float64(1.3600022280588746) - rollout_corr/rollout_log_ppl:np.float64(0.30314545703004114) - rollout_corr/log_ppl_diff:np.float64(0.0010337693965993822) - rollout_corr/log_ppl_abs_diff:np.float64(0.002399776072707027) - rollout_corr/log_ppl_diff_max:np.float64(0.011399328708648682) - rollout_corr/log_ppl_diff_min:np.float64(-0.006810277700424194) - rollout_corr/ppl_ratio:np.float64(1.0010383201297373) - rollout_corr/chi2_token:np.float64(0.0029240259900689125) - rollout_corr/chi2_seq:np.float64(1.0353804978076369) - actor/pg_loss:np.float64(0.007192879682406783) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010274524142914743) - actor/ppo_kl:np.float64(4.0873823671461196e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.64453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.42578125) - self_distillation/token_reweight_w_mean:np.float64(90371.76209130394) - self_distillation/token_reweight_w_std:np.float64(1389709.4737052019) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0000855000689626) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11968303663888946) - self_distillation/empty_target_batch:np.float64(0.35546875) - actor/grad_norm:np.float64(0.6043514311313629) - perf/mfu/actor:np.float64(0.03566211221390109) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.67253494262695) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.42578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.42578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006435959134250879 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006435959134250879 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:327.82421875 - response_length/max:698.0 - response_length/min:141.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:327.82421875 - response_length_non_aborted/max:698.0 - response_length_non_aborted/min:141.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.71875 - prompt_length/max:859.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010755844414234161 - timing_s/agent_loop/generate_sequences/min:np.float64(1.786530127748847) - timing_s/agent_loop/generate_sequences/max:np.float64(5.909618901088834) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.657506644296518) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.909618901088834) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:670 - timing_s/gen:11.721529113128781 - timing_s/reward:0.05861745588481426 - timing_s/old_log_prob:2.29558939486742 - timing_s/adv:0.5112862680107355 - timing_s/update_actor:15.580748109146953 - timing_s/step:30.256891636177897 - timing_s/stop_profile:0.00016321614384651184 - timing_per_token_ms/adv:0.003281957210875975 - timing_per_token_ms/gen:0.13967004412531464 - timing_per_token_ms/update_actor:0.10001314685530213 - perf/total_num_tokens:155787 - perf/time_per_step:30.256891636177897 - perf/throughput:643.6013069074107 (TaskRunner pid=2026458) Training Progress: 16%|█▌ | 16/100 [11:56<50:48, 36.29s/it] (TaskRunner pid=2026458) step:17 - global_seqlen/min:19903 - global_seqlen/max:25273 - global_seqlen/minmax_diff:5370 - global_seqlen/balanced_min:22184 - global_seqlen/balanced_max:22190 - global_seqlen/mean:22187.375 - actor/entropy:0.28030383586883545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4096241295337677 - training/rollout_probs_diff_mean:0.00557515537366271 - training/rollout_probs_diff_std:0.014061576686799526 - training/rollout_actor_probs_pearson_corr:0.9982938766479492 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.640625 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.640625 - rollout_corr/rollout_is_mean:1.000001311302185 - rollout_corr/rollout_is_ratio_fraction_high:2.1693151211366057e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.4232874390436336e-05 - rollout_corr/rollout_is_max:3.007925271987915 - rollout_corr/rollout_is_min:0.3316154181957245 - rollout_corr/rollout_is_std:0.0431625172495842 - rollout_corr/rollout_is_eff_sample_size:0.9981403427818611 - rollout_corr/rollout_is_seq_mean:1.0001276731491089 - rollout_corr/rollout_is_seq_std:0.002614374505355954 - rollout_corr/rollout_is_seq_max:1.0077310800552368 - rollout_corr/rollout_is_seq_min:0.9923279881477356 - rollout_corr/rollout_is_seq_max_deviation:0.007731080055236816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.381326341535896) - rollout_corr/training_log_ppl:np.float64(0.3175459819030948) - rollout_corr/kl:np.float64(0.0007249800520758853) - rollout_corr/k3_kl:np.float64(0.0011385135364321286) - rollout_corr/rollout_ppl:np.float64(1.3803315898403525) - rollout_corr/rollout_log_ppl:np.float64(0.316821001295466) - rollout_corr/log_ppl_diff:np.float64(0.0007249806076288223) - rollout_corr/log_ppl_abs_diff:np.float64(0.00215923844370991) - rollout_corr/log_ppl_diff_max:np.float64(0.00994020700454712) - rollout_corr/log_ppl_diff_min:np.float64(-0.007171809673309326) - rollout_corr/ppl_ratio:np.float64(1.0007290567737073) - rollout_corr/chi2_token:np.float64(0.003144999034702778) - rollout_corr/chi2_seq:np.float64(0.9032414378598332) - actor/pg_loss:np.float64(0.0055801779963076115) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008305092796945246) - actor/ppo_kl:np.float64(-0.00022399483597723702) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.640625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(213885.15984977316) - self_distillation/token_reweight_w_std:np.float64(3187382.02780142) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009852752555162) - self_distillation/token_reweight_w_clip_frac:np.float64(0.11256717666401528) - self_distillation/empty_target_batch:np.float64(0.359375) - actor/grad_norm:np.float64(0.5271423608064651) - perf/mfu/actor:np.float64(0.04032452755142399) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.51458740234375) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0036620749160647392 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0036620749160647392 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:360.13671875 - response_length/max:1140.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:360.13671875 - response_length_non_aborted/max:1140.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:333.21875 - prompt_length/max:617.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014656782150268555 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6168197989463806) - timing_s/agent_loop/generate_sequences/max:np.float64(8.074994288384914) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7389801555254962) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.074994288384914) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1140 - timing_s/gen:13.71630746126175 - timing_s/reward:0.06177904084324837 - timing_s/old_log_prob:2.311936927959323 - timing_s/adv:0.5291431322693825 - timing_s/update_actor:15.853100046515465 - timing_s/step:32.560671512037516 - timing_s/stop_profile:0.00012019649147987366 - timing_per_token_ms/adv:0.0029811048640802622 - timing_per_token_ms/gen:0.14877496026098758 - timing_per_token_ms/update_actor:0.08931374287469487 - perf/total_num_tokens:177499 - perf/time_per_step:32.560671512037516 - perf/throughput:681.4163826995226 (TaskRunner pid=2026458) Training Progress: 17%|█▋ | 17/100 [12:29<48:40, 35.18s/it] (TaskRunner pid=2026458) step:18 - global_seqlen/min:17147 - global_seqlen/max:20375 - global_seqlen/minmax_diff:3228 - global_seqlen/balanced_min:18455 - global_seqlen/balanced_max:18458 - global_seqlen/mean:18457.0 - actor/entropy:0.3130648136138916 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2772509455680847 - training/rollout_probs_diff_mean:0.0063330200500786304 - training/rollout_probs_diff_std:0.014608096331357956 - training/rollout_actor_probs_pearson_corr:0.9982773065567017 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73046875 - self_distillation/correct_sample_fraction:0.421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73046875 - rollout_corr/rollout_is_mean:0.9997703433036804 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.43533785478212e-05 - rollout_corr/rollout_is_max:1.9900730848312378 - rollout_corr/rollout_is_min:0.4134674072265625 - rollout_corr/rollout_is_std:0.04487474635243416 - rollout_corr/rollout_is_eff_sample_size:0.9979895323387393 - rollout_corr/rollout_is_seq_mean:0.9998025298118591 - rollout_corr/rollout_is_seq_std:0.002652442781254649 - rollout_corr/rollout_is_seq_max:1.0070677995681763 - rollout_corr/rollout_is_seq_min:0.9913365244865417 - rollout_corr/rollout_is_seq_max_deviation:0.008663475513458252 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3969450276345015) - rollout_corr/training_log_ppl:np.float64(0.3299181046313606) - rollout_corr/kl:np.float64(0.001080212333377517) - rollout_corr/k3_kl:np.float64(0.0011240256234259505) - rollout_corr/rollout_ppl:np.float64(1.3953958731144667) - rollout_corr/rollout_log_ppl:np.float64(0.32883789180777967) - rollout_corr/log_ppl_diff:np.float64(0.0010802128235809505) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023538178065791726) - rollout_corr/log_ppl_diff_max:np.float64(0.011510014533996582) - rollout_corr/log_ppl_diff_min:np.float64(-0.009586125612258911) - rollout_corr/ppl_ratio:np.float64(1.0010849123355001) - rollout_corr/chi2_token:np.float64(0.0023459065705537796) - rollout_corr/chi2_seq:np.float64(0.5731152268126607) - actor/pg_loss:np.float64(0.007650406565517187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011541137814674585) - actor/ppo_kl:np.float64(-0.00018551408525979696) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.421875) - self_distillation/token_reweight_w_mean:np.float64(258180.61296664365) - self_distillation/token_reweight_w_std:np.float64(3445719.439609512) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022555205505341) - self_distillation/token_reweight_w_clip_frac:np.float64(0.15918678505113348) - self_distillation/empty_target_batch:np.float64(0.26953125) - actor/grad_norm:np.float64(0.6539853662252426) - perf/mfu/actor:np.float64(0.033737344150768206) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.57009506225586) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005579437594860792 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005579437594860792 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:303.5 - response_length/max:713.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:303.5 - response_length_non_aborted/max:713.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.28125 - prompt_length/max:540.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013168901205062866 - timing_s/agent_loop/generate_sequences/min:np.float64(1.151022594422102) - timing_s/agent_loop/generate_sequences/max:np.float64(5.789204815402627) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2287484776461497) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.789204815402627) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:713 - timing_s/gen:11.655897682532668 - timing_s/reward:0.05685456842184067 - timing_s/old_log_prob:2.3437488954514265 - timing_s/adv:0.4995372127741575 - timing_s/update_actor:15.640153270214796 - timing_s/step:30.285941103473306 - timing_s/stop_profile:0.00014618411660194397 - timing_per_token_ms/adv:0.0033831148939031095 - timing_per_token_ms/gen:0.15001927618580968 - timing_per_token_ms/update_actor:0.10592291048257298 - perf/total_num_tokens:147656 - perf/time_per_step:30.285941103473306 - perf/throughput:609.4246811397015 (TaskRunner pid=2026458) Training Progress: 18%|█▊ | 18/100 [12:59<46:05, 33.73s/it] (TaskRunner pid=2026458) step:19 - global_seqlen/min:16712 - global_seqlen/max:22903 - global_seqlen/minmax_diff:6191 - global_seqlen/balanced_min:19739 - global_seqlen/balanced_max:19742 - global_seqlen/mean:19740.875 - actor/entropy:0.30067509412765503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2716989815235138 - training/rollout_probs_diff_mean:0.006011276505887508 - training/rollout_probs_diff_std:0.014068786054849625 - training/rollout_actor_probs_pearson_corr:0.9983283281326294 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0000412464141846 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.48191299760947e-05 - rollout_corr/rollout_is_max:1.967125415802002 - rollout_corr/rollout_is_min:0.3074234426021576 - rollout_corr/rollout_is_std:0.04442286118865013 - rollout_corr/rollout_is_eff_sample_size:0.9980306147615733 - rollout_corr/rollout_is_seq_mean:1.0000803470611572 - rollout_corr/rollout_is_seq_std:0.0029041871894150972 - rollout_corr/rollout_is_seq_max:1.0110079050064087 - rollout_corr/rollout_is_seq_min:0.9903981685638428 - rollout_corr/rollout_is_seq_max_deviation:0.011007905006408691 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.375831812620163) - rollout_corr/training_log_ppl:np.float64(0.31493956671329215) - rollout_corr/kl:np.float64(0.0009361723424987645) - rollout_corr/k3_kl:np.float64(0.00108607468507671) - rollout_corr/rollout_ppl:np.float64(1.3745251446962357) - rollout_corr/rollout_log_ppl:np.float64(0.31400339302490465) - rollout_corr/log_ppl_diff:np.float64(0.0009361736883874983) - rollout_corr/log_ppl_abs_diff:np.float64(0.002310327283339575) - rollout_corr/log_ppl_diff_max:np.float64(0.008226662874221802) - rollout_corr/log_ppl_diff_min:np.float64(-0.010857582092285156) - rollout_corr/ppl_ratio:np.float64(1.000940806698054) - rollout_corr/chi2_token:np.float64(0.002460781019181013) - rollout_corr/chi2_seq:np.float64(1.263379932148382) - actor/pg_loss:np.float64(0.005378377623856068) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007215913469735824) - actor/ppo_kl:np.float64(-2.377988029422795e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(92620.61167126987) - self_distillation/token_reweight_w_std:np.float64(1450653.645121337) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001238340511918) - self_distillation/token_reweight_w_clip_frac:np.float64(0.096618483774364) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.5270146429538727) - perf/mfu/actor:np.float64(0.03604524222111994) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.61618423461914) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0024989761877804995 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0024989761877804995 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:314.77734375 - response_length/max:828.0 - response_length/min:154.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:314.77734375 - response_length_non_aborted/max:828.0 - response_length_non_aborted/min:154.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.125 - prompt_length/max:836.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014412589371204376 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8398964125663042) - timing_s/agent_loop/generate_sequences/max:np.float64(6.303931085392833) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4352486536299693) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.303931085392833) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:828 - timing_s/gen:12.11710455827415 - timing_s/reward:0.05806134268641472 - timing_s/old_log_prob:2.3140441831201315 - timing_s/adv:0.5250083282589912 - timing_s/update_actor:15.580280201509595 - timing_s/step:30.68251054175198 - timing_s/stop_profile:0.00012233667075634003 - timing_per_token_ms/adv:0.0033243734653288625 - timing_per_token_ms/gen:0.150368000177136 - timing_per_token_ms/update_actor:0.09865494944822352 - perf/total_num_tokens:157927 - perf/time_per_step:30.68251054175198 - perf/throughput:643.3917776427427 (TaskRunner pid=2026458) Training Progress: 19%|█▉ | 19/100 [13:30<44:19, 32.83s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which molecule is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". This product contains a structure with a substituted benzene ring (with a Cl group), a trifluoromethyl group (C(F)(F)F), and a nitrogen-containing amine group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A contains a structure with a substituted benzene ring, a nitrogen atom, and a carbonyl group. It also includes the Cl and trifluoromethyl groups, which match the product's structure. However, it has an extra nitrogen and a different arrangement that may not be the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option B has a similar structure but with a different arrangement of substituents and a different nitrogen configuration, which does not match the product's structure exactly. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option C includes a substituted benzene ring, a carbonyl group, and a different arrangement of substituents. It includes a hydroxyl group and a different type of substitution, which does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option D has a substituted benzene ring, a carbonyl group, and a different arrangement of substituents. It includes a different type of substitution and does not match the product's structure exactly. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A is the only one that matches the structure of the product, including the correct substituents and arrangement of atoms. Therefore, it is the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2026458) step:20 - global_seqlen/min:17486 - global_seqlen/max:22655 - global_seqlen/minmax_diff:5169 - global_seqlen/balanced_min:20031 - global_seqlen/balanced_max:20228 - global_seqlen/mean:20058.625 - actor/entropy:0.297139972448349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3528301417827606 - training/rollout_probs_diff_mean:0.006166450679302216 - training/rollout_probs_diff_std:0.014436931349337101 - training/rollout_actor_probs_pearson_corr:0.9982419013977051 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.8359375 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8359375 - rollout_corr/rollout_is_mean:1.000038743019104 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.690288890036754e-05 - rollout_corr/rollout_is_max:1.9574382305145264 - rollout_corr/rollout_is_min:0.3535598814487457 - rollout_corr/rollout_is_std:0.04514159634709358 - rollout_corr/rollout_is_eff_sample_size:0.9979664991652689 - rollout_corr/rollout_is_seq_mean:1.0000238418579102 - rollout_corr/rollout_is_seq_std:0.0029090906027704477 - rollout_corr/rollout_is_seq_max:1.0110317468643188 - rollout_corr/rollout_is_seq_min:0.9888052940368652 - rollout_corr/rollout_is_seq_max_deviation:0.011194705963134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3683372386731207) - rollout_corr/training_log_ppl:np.float64(0.31030508255935274) - rollout_corr/kl:np.float64(0.0011023225397535086) - rollout_corr/k3_kl:np.float64(0.001434143366623175) - rollout_corr/rollout_ppl:np.float64(1.3668043981306255) - rollout_corr/rollout_log_ppl:np.float64(0.30920275836251676) - rollout_corr/log_ppl_diff:np.float64(0.0011023241968359798) - rollout_corr/log_ppl_abs_diff:np.float64(0.002514752763090655) - rollout_corr/log_ppl_diff_max:np.float64(0.0131492018699646) - rollout_corr/log_ppl_diff_min:np.float64(-0.008862733840942383) - rollout_corr/ppl_ratio:np.float64(1.0011082254350185) - rollout_corr/chi2_token:np.float64(0.0035093913320451975) - rollout_corr/chi2_seq:np.float64(1.2149556565564126) - actor/pg_loss:np.float64(0.009395288652740419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015388868646368792) - actor/ppo_kl:np.float64(4.407900779046514e-05) - actor/pg_clipfrac_lower:np.float64(3.007869418070186e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8359375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8359375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(178377.74242206803) - self_distillation/token_reweight_w_std:np.float64(2601166.713610011) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9996962018776685) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1573005250247661) - self_distillation/empty_target_batch:np.float64(0.1640625) - actor/grad_norm:np.float64(0.68055609613657) - perf/mfu/actor:np.float64(0.03693197062950951) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.50461959838867) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1512467176955187) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7826571428571428) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06785839628022725) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.5356190476190476) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.11104188480484187) - val-aux/sciknoweval/score/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/score/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/score/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1512467176955187) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7826571428571428) - val-aux/sciknoweval/score/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06785839628022725) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.5356190476190476) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.11104188480484187) - val-core/sciknoweval/acc/mean@16:np.float64(0.5101190476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.24920759452591995) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6139380952380953) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.2043060724638079) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.40814285714285714) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.21773506231934722) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5119238095238096) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.24829066308997053) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.6943) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.144133820391533) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.32112857142857143) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1656815962127734) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.5277333333333334) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.20232169992437612) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7490619047619047) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09145129929830753) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.2561380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.11226817567713847) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.5346095238095239) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1512467176955187) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7826571428571428) - val-core/sciknoweval/acc/best@16/std:np.float64(0.051125776744021086) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.21173809523809525) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06785839628022725) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.5356190476190476) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.11104188480484187) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999809523809524) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.000300566358820181) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9992904761904762) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0016956593912030376) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9996047619047619) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0013137248630794105) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9988428571428571) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.002042357581276089) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999285714285715) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0005788212276734866) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980380952380952) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023437861108329258) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9968761904761905) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.002262100242179584) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011333633214235306 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011333633214235306 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:343.23828125 - response_length/max:1220.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:343.23828125 - response_length_non_aborted/max:1220.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:392.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000131312757730484 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3023900520056486) - timing_s/agent_loop/generate_sequences/max:np.float64(8.421228867024183) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.7250610580740613) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.421228867024183) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:1220 - timing_s/gen:14.119003845378757 - timing_s/reward:0.058908019214868546 - timing_s/old_log_prob:2.3049094192683697 - timing_s/adv:0.5378871113061905 - timing_s/update_actor:15.319367559626698 - timing_s/step:32.428459821268916 - timing_s/testing:112.78920979425311 - timing_s/save_checkpoint:6.872485619038343 - timing_s/stop_profile:0.00016718730330467224 - timing_per_token_ms/adv:0.003351968986571802 - timing_per_token_ms/gen:0.16068242321386106 - timing_per_token_ms/update_actor:0.09546621191399397 - perf/total_num_tokens:160469 - perf/time_per_step:32.428459821268916 - perf/throughput:618.5500363123663 (TaskRunner pid=2026458) Training Progress: 20%|██ | 20/100 [16:02<1:31:32, 68.66s/it] (TaskRunner pid=2026458) step:21 - global_seqlen/min:17387 - global_seqlen/max:23009 - global_seqlen/minmax_diff:5622 - global_seqlen/balanced_min:19296 - global_seqlen/balanced_max:19303 - global_seqlen/mean:19300.625 - actor/entropy:0.2956748902797699 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42548468708992004 - training/rollout_probs_diff_mean:0.005995167884975672 - training/rollout_probs_diff_std:0.014325084164738655 - training/rollout_actor_probs_pearson_corr:0.9982481002807617 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:0.999878466129303 - rollout_corr/rollout_is_ratio_fraction_high:1.2208670341351535e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.325202022911981e-05 - rollout_corr/rollout_is_max:2.6692886352539062 - rollout_corr/rollout_is_min:0.4042225778102875 - rollout_corr/rollout_is_std:0.044611647725105286 - rollout_corr/rollout_is_eff_sample_size:0.9980132789914753 - rollout_corr/rollout_is_seq_mean:0.9997992515563965 - rollout_corr/rollout_is_seq_std:0.0029755644500255585 - rollout_corr/rollout_is_seq_max:1.0099884271621704 - rollout_corr/rollout_is_seq_min:0.9911137819290161 - rollout_corr/rollout_is_seq_max_deviation:0.00998842716217041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3673860565759242) - rollout_corr/training_log_ppl:np.float64(0.3087902419210877) - rollout_corr/kl:np.float64(0.002253606636003269) - rollout_corr/k3_kl:np.float64(0.0017760526020538236) - rollout_corr/rollout_ppl:np.float64(1.36424843640998) - rollout_corr/rollout_log_ppl:np.float64(0.30653663334669545) - rollout_corr/log_ppl_diff:np.float64(0.002253608574392274) - rollout_corr/log_ppl_abs_diff:np.float64(0.0032565151050221175) - rollout_corr/log_ppl_diff_max:np.float64(0.021895110607147217) - rollout_corr/log_ppl_diff_min:np.float64(-0.006019324064254761) - rollout_corr/ppl_ratio:np.float64(1.002263783942908) - rollout_corr/chi2_token:np.float64(0.002287261188030243) - rollout_corr/chi2_seq:np.float64(0.5934691540896893) - actor/pg_loss:np.float64(0.010032993159256876) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001869809015715873) - actor/ppo_kl:np.float64(0.000984730247550747) - actor/pg_clipfrac_lower:np.float64(2.9235126021376345e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(353075.78875684226) - self_distillation/token_reweight_w_std:np.float64(4698832.619528098) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0033803705591708) - self_distillation/token_reweight_w_clip_frac:np.float64(0.17246504218201153) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.7826535254716873) - perf/mfu/actor:np.float64(0.03582340340916603) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.28646850585938) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004340182524174452 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004340182524174452 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:319.95703125 - response_length/max:825.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:319.95703125 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.1875 - prompt_length/max:669.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.8999285101890564e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3953026235103607) - timing_s/agent_loop/generate_sequences/max:np.float64(6.321475323289633) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4004087638677447) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.321475323289633) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:348 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:12.068734215572476 - timing_s/reward:0.05136189982295036 - timing_s/old_log_prob:2.455154914408922 - timing_s/adv:0.532898997887969 - timing_s/update_actor:15.235047852620482 - timing_s/step:30.448000945150852 - timing_s/stop_profile:0.00011887960135936737 - timing_per_token_ms/adv:0.003451306614992837 - timing_per_token_ms/gen:0.14734320057102976 - timing_per_token_ms/update_actor:0.09866939446663309 - perf/total_num_tokens:154405 - perf/time_per_step:30.448000945150852 - perf/throughput:633.8880846321642 (TaskRunner pid=2026458) Training Progress: 21%|██ | 21/100 [16:33<1:15:19, 57.20s/it] (TaskRunner pid=2026458) step:22 - global_seqlen/min:15143 - global_seqlen/max:18708 - global_seqlen/minmax_diff:3565 - global_seqlen/balanced_min:16965 - global_seqlen/balanced_max:16966 - global_seqlen/mean:16965.375 - actor/entropy:0.31038597226142883 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2530360817909241 - training/rollout_probs_diff_mean:0.0064188591204583645 - training/rollout_probs_diff_std:0.014548459090292454 - training/rollout_actor_probs_pearson_corr:0.9982183575630188 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.66796875 - self_distillation/correct_sample_fraction:0.375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.66796875 - rollout_corr/rollout_is_mean:1.0001835823059082 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.267485928721726e-05 - rollout_corr/rollout_is_max:1.9917649030685425 - rollout_corr/rollout_is_min:0.3914262652397156 - rollout_corr/rollout_is_std:0.0448860339820385 - rollout_corr/rollout_is_eff_sample_size:0.9979898885301821 - rollout_corr/rollout_is_seq_mean:1.0002237558364868 - rollout_corr/rollout_is_seq_std:0.0030402897391468287 - rollout_corr/rollout_is_seq_max:1.0082043409347534 - rollout_corr/rollout_is_seq_min:0.9909875988960266 - rollout_corr/rollout_is_seq_max_deviation:0.009012401103973389 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3948297398164868) - rollout_corr/training_log_ppl:np.float64(0.32873041287530214) - rollout_corr/kl:np.float64(0.0012066373147963105) - rollout_corr/k3_kl:np.float64(0.0013965993947522293) - rollout_corr/rollout_ppl:np.float64(1.3931263061240315) - rollout_corr/rollout_log_ppl:np.float64(0.32752377312863246) - rollout_corr/log_ppl_diff:np.float64(0.0012066397466696799) - rollout_corr/log_ppl_abs_diff:np.float64(0.002761501877103001) - rollout_corr/log_ppl_diff_max:np.float64(0.011306077241897583) - rollout_corr/log_ppl_diff_min:np.float64(-0.009509235620498657) - rollout_corr/ppl_ratio:np.float64(1.0012132688425481) - rollout_corr/chi2_token:np.float64(0.003218145342543721) - rollout_corr/chi2_seq:np.float64(1.0662864169571549) - actor/pg_loss:np.float64(0.006237241672351956) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020408303180374787) - actor/ppo_kl:np.float64(0.0003451099556370796) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.66796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.375) - self_distillation/token_reweight_w_mean:np.float64(326844.6732725387) - self_distillation/token_reweight_w_std:np.float64(4298094.462517299) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0025643010158092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1402466203435324) - self_distillation/empty_target_batch:np.float64(0.33203125) - actor/grad_norm:np.float64(0.6192770302295685) - perf/mfu/actor:np.float64(0.03161979996245398) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.7820701599121) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003102818038314581 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003102818038314581 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.60546875 - response_length/max:646.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.60546875 - response_length_non_aborted/max:646.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:592.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012409500777721405 - timing_s/agent_loop/generate_sequences/min:np.float64(1.460020076483488) - timing_s/agent_loop/generate_sequences/max:np.float64(5.160831518471241) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.023891647448181) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.160831518471241) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:646 - timing_s/gen:10.945641493424773 - timing_s/reward:0.05562561750411987 - timing_s/old_log_prob:2.356655828654766 - timing_s/adv:0.4728220421820879 - timing_s/update_actor:15.27205784060061 - timing_s/step:29.190949207171798 - timing_s/stop_profile:0.0001829788088798523 - timing_per_token_ms/adv:0.003483728197741635 - timing_per_token_ms/gen:0.15570124032240534 - timing_per_token_ms/update_actor:0.11252372730193563 - perf/total_num_tokens:135723 - perf/time_per_step:29.190949207171798 - perf/throughput:581.1861368259944 (TaskRunner pid=2026458) Training Progress: 22%|██▏ | 22/100 [17:02<1:03:27, 48.81s/it] (TaskRunner pid=2026458) step:23 - global_seqlen/min:15206 - global_seqlen/max:25332 - global_seqlen/minmax_diff:10126 - global_seqlen/balanced_min:18411 - global_seqlen/balanced_max:25520 - global_seqlen/mean:19300.375 - actor/entropy:0.264564573764801 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4770219326019287 - training/rollout_probs_diff_mean:0.0052871485240757465 - training/rollout_probs_diff_std:0.013520004227757454 - training/rollout_actor_probs_pearson_corr:0.998356282711029 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999374747276306 - rollout_corr/rollout_is_ratio_fraction_high:3.601138087105937e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.0018966905772686e-05 - rollout_corr/rollout_is_max:2.1046066284179688 - rollout_corr/rollout_is_min:0.36882883310317993 - rollout_corr/rollout_is_std:0.041905514895915985 - rollout_corr/rollout_is_eff_sample_size:0.9982467687554272 - rollout_corr/rollout_is_seq_mean:0.9998911023139954 - rollout_corr/rollout_is_seq_std:0.0030532306991517544 - rollout_corr/rollout_is_seq_max:1.0095295906066895 - rollout_corr/rollout_is_seq_min:0.9908995628356934 - rollout_corr/rollout_is_seq_max_deviation:0.009529590606689453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3668543784879148) - rollout_corr/training_log_ppl:np.float64(0.30833037884258374) - rollout_corr/kl:np.float64(0.0014712483040568713) - rollout_corr/k3_kl:np.float64(0.0014252641368557306) - rollout_corr/rollout_ppl:np.float64(1.364856572356075) - rollout_corr/rollout_log_ppl:np.float64(0.30685912866283616) - rollout_corr/log_ppl_diff:np.float64(0.0014712501797475852) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029776765513815917) - rollout_corr/log_ppl_diff_max:np.float64(0.01450282335281372) - rollout_corr/log_ppl_diff_min:np.float64(-0.008856803178787231) - rollout_corr/ppl_ratio:np.float64(1.0014787362888455) - rollout_corr/chi2_token:np.float64(0.003022435586899519) - rollout_corr/chi2_seq:np.float64(1.3533644466660917) - actor/pg_loss:np.float64(0.007422412978485227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0020335467470431468) - actor/ppo_kl:np.float64(0.00035265562636155323) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(345985.51773530385) - self_distillation/token_reweight_w_std:np.float64(4731856.136310147) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988346283789724) - self_distillation/token_reweight_w_clip_frac:np.float64(0.16046884056413546) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.7161731868982315) - perf/mfu/actor:np.float64(0.03389860046243484) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1311912536621) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008347137831151485 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008347137831151485 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.41796875 - response_length/max:8192.0 - response_length/min:126.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:325.41796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:277.71875 - prompt_length/max:766.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010981783270835876 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5618780367076397) - timing_s/agent_loop/generate_sequences/max:np.float64(49.69344619102776) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.4613353605309385) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.69344619102776) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:230 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.576194578781724 - timing_s/reward:0.05657547526061535 - timing_s/old_log_prob:2.56242498382926 - timing_s/adv:0.5329509787261486 - timing_s/update_actor:16.533158656209707 - timing_s/step:75.34719325602055 - timing_s/stop_profile:0.00011706724762916565 - timing_per_token_ms/adv:0.0034516879770869 - timing_per_token_ms/gen:0.6671251464916721 - timing_per_token_ms/update_actor:0.1070779625798055 - perf/total_num_tokens:154403 - perf/time_per_step:75.34719325602055 - perf/throughput:256.1525408706292 (TaskRunner pid=2026458) Training Progress: 23%|██▎ | 23/100 [18:17<1:12:52, 56.79s/it] (TaskRunner pid=2026458) step:24 - global_seqlen/min:16231 - global_seqlen/max:19060 - global_seqlen/minmax_diff:2829 - global_seqlen/balanced_min:17704 - global_seqlen/balanced_max:17708 - global_seqlen/mean:17706.25 - actor/entropy:0.29770269989967346 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5271099209785461 - training/rollout_probs_diff_mean:0.0061988383531570435 - training/rollout_probs_diff_std:0.014574181288480759 - training/rollout_actor_probs_pearson_corr:0.9981999397277832 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:1.000082015991211 - rollout_corr/rollout_is_ratio_fraction_high:2.8622949685086496e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011449179874034598 - rollout_corr/rollout_is_max:2.555720567703247 - rollout_corr/rollout_is_min:0.15318159759044647 - rollout_corr/rollout_is_std:0.04580353945493698 - rollout_corr/rollout_is_eff_sample_size:0.997906784084707 - rollout_corr/rollout_is_seq_mean:1.000064492225647 - rollout_corr/rollout_is_seq_std:0.0029369699768722057 - rollout_corr/rollout_is_seq_max:1.0110825300216675 - rollout_corr/rollout_is_seq_min:0.9918853640556335 - rollout_corr/rollout_is_seq_max_deviation:0.01108253002166748 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.36986547941342) - rollout_corr/training_log_ppl:np.float64(0.31092395351151936) - rollout_corr/kl:np.float64(0.0015822704315340275) - rollout_corr/k3_kl:np.float64(0.0015203862882344765) - rollout_corr/rollout_ppl:np.float64(1.3676151768304408) - rollout_corr/rollout_log_ppl:np.float64(0.3093416829360649) - rollout_corr/log_ppl_diff:np.float64(0.0015822705754544586) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029158030229154974) - rollout_corr/log_ppl_diff_max:np.float64(0.013341128826141357) - rollout_corr/log_ppl_diff_min:np.float64(-0.009624361991882324) - rollout_corr/ppl_ratio:np.float64(1.0015899618156254) - rollout_corr/chi2_token:np.float64(0.0028670839965343475) - rollout_corr/chi2_seq:np.float64(0.6717999817337841) - actor/pg_loss:np.float64(0.0065518481424078345) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001698189102626202) - actor/ppo_kl:np.float64(0.000539918119613958) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(259653.62712082197) - self_distillation/token_reweight_w_std:np.float64(3592355.994360844) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997991549316794) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1357375134830363) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.6166152656078339) - perf/mfu/actor:np.float64(0.03283040904184733) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1153450012207) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003980379085987806 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003980379085987806 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.9453125 - response_length/max:645.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.9453125 - response_length_non_aborted/max:645.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.375 - prompt_length/max:848.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011000782251358032 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4089452624320984) - timing_s/agent_loop/generate_sequences/max:np.float64(5.263506431132555) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0320224383394816) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.263506431132555) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:645 - timing_s/gen:11.14209609106183 - timing_s/reward:0.05296533182263374 - timing_s/old_log_prob:2.2817067373543978 - timing_s/adv:0.506876740604639 - timing_s/update_actor:15.04843096062541 - timing_s/step:29.098722390830517 - timing_s/stop_profile:0.00011877715587615967 - timing_per_token_ms/adv:0.003578374448320784 - timing_per_token_ms/gen:0.15945982899307082 - timing_per_token_ms/update_actor:0.10623671698288323 - perf/total_num_tokens:141650 - perf/time_per_step:29.098722390830517 - perf/throughput:608.4889144679262 (TaskRunner pid=2026458) Training Progress: 24%|██▍ | 24/100 [18:46<1:01:25, 48.49s/it] (TaskRunner pid=2026458) step:25 - global_seqlen/min:15795 - global_seqlen/max:22433 - global_seqlen/minmax_diff:6638 - global_seqlen/balanced_min:18523 - global_seqlen/balanced_max:18527 - global_seqlen/mean:18525.875 - actor/entropy:0.2758452296257019 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.25533485412597656 - training/rollout_probs_diff_mean:0.005769852548837662 - training/rollout_probs_diff_std:0.013884530402719975 - training/rollout_actor_probs_pearson_corr:0.9983004331588745 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.78515625 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78515625 - rollout_corr/rollout_is_mean:1.0001115798950195 - rollout_corr/rollout_is_ratio_fraction_high:1.414367125107674e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.828734250215348e-05 - rollout_corr/rollout_is_max:2.6026298999786377 - rollout_corr/rollout_is_min:0.3732539415359497 - rollout_corr/rollout_is_std:0.04336366429924965 - rollout_corr/rollout_is_eff_sample_size:0.9981233594863631 - rollout_corr/rollout_is_seq_mean:1.000120759010315 - rollout_corr/rollout_is_seq_std:0.003118946449831128 - rollout_corr/rollout_is_seq_max:1.0095044374465942 - rollout_corr/rollout_is_seq_min:0.990989625453949 - rollout_corr/rollout_is_seq_max_deviation:0.009504437446594238 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3535170801915228) - rollout_corr/training_log_ppl:np.float64(0.2983562831650488) - rollout_corr/kl:np.float64(0.000989430469498842) - rollout_corr/k3_kl:np.float64(0.0011073322453967194) - rollout_corr/rollout_ppl:np.float64(1.3521247510798275) - rollout_corr/rollout_log_ppl:np.float64(0.29736685164971277) - rollout_corr/log_ppl_diff:np.float64(0.0009894315153360367) - rollout_corr/log_ppl_abs_diff:np.float64(0.002447564620524645) - rollout_corr/log_ppl_diff_max:np.float64(0.01045176386833191) - rollout_corr/log_ppl_diff_min:np.float64(-0.008554220199584961) - rollout_corr/ppl_ratio:np.float64(1.0009945430792868) - rollout_corr/chi2_token:np.float64(0.002472628140822053) - rollout_corr/chi2_seq:np.float64(0.6732058441266418) - actor/pg_loss:np.float64(0.005739172571338713) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010865399485737726) - actor/ppo_kl:np.float64(0.0001121951680964628) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78515625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78515625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(393425.2932063611) - self_distillation/token_reweight_w_std:np.float64(5274741.3204780845) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0018663464579731) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12906738638412207) - self_distillation/empty_target_batch:np.float64(0.21484375) - actor/grad_norm:np.float64(0.6542618125677109) - perf/mfu/actor:np.float64(0.034142301102186606) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1241111755371) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:0 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005874927621334791 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005874927621334791 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:276.18359375 - response_length/max:807.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:276.18359375 - response_length_non_aborted/max:807.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.75 - prompt_length/max:646.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014953315258026123 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3853862024843693) - timing_s/agent_loop/generate_sequences/max:np.float64(5.845100851729512) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.959879736532457) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.845100851729512) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:213 - timing_s/agent_loop/slowest/response_length:807 - timing_s/gen:11.540503876283765 - timing_s/reward:0.05447870306670666 - timing_s/old_log_prob:2.3400896415114403 - timing_s/adv:0.5113725643604994 - timing_s/update_actor:15.447554072365165 - timing_s/step:29.938440999016166 - timing_s/stop_profile:0.00012017786502838135 - timing_per_token_ms/adv:0.0034503941403611123 - timing_per_token_ms/gen:0.16322509478075561 - timing_per_token_ms/update_actor:0.10422958478590866 - perf/total_num_tokens:148207 - perf/time_per_step:29.938440999016166 - perf/throughput:618.7989214471387 (TaskRunner pid=2026458) Training Progress: 25%|██▌ | 25/100 [19:16<53:40, 42.94s/it] (TaskRunner pid=2026458) step:26 - global_seqlen/min:16232 - global_seqlen/max:20091 - global_seqlen/minmax_diff:3859 - global_seqlen/balanced_min:18014 - global_seqlen/balanced_max:18018 - global_seqlen/mean:18016.5 - actor/entropy:0.28236204385757446 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797455430030823 - training/rollout_probs_diff_mean:0.005817539524286985 - training/rollout_probs_diff_std:0.014095457270741463 - training/rollout_actor_probs_pearson_corr:0.9982751607894897 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000920295715332 - rollout_corr/rollout_is_ratio_fraction_high:1.4651584933744743e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.790950960246846e-05 - rollout_corr/rollout_is_max:2.0633111000061035 - rollout_corr/rollout_is_min:0.2806716561317444 - rollout_corr/rollout_is_std:0.044501952826976776 - rollout_corr/rollout_is_eff_sample_size:0.9980238466152811 - rollout_corr/rollout_is_seq_mean:1.0001037120819092 - rollout_corr/rollout_is_seq_std:0.0033811419270932674 - rollout_corr/rollout_is_seq_max:1.010282039642334 - rollout_corr/rollout_is_seq_min:0.9834394454956055 - rollout_corr/rollout_is_seq_max_deviation:0.01656055450439453 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3481695111840963) - rollout_corr/training_log_ppl:np.float64(0.294715576863382) - rollout_corr/kl:np.float64(0.00122398861193318) - rollout_corr/k3_kl:np.float64(0.0011840371912512637) - rollout_corr/rollout_ppl:np.float64(1.3464941019192338) - rollout_corr/rollout_log_ppl:np.float64(0.2934915872756392) - rollout_corr/log_ppl_diff:np.float64(0.0012239895877428353) - rollout_corr/log_ppl_abs_diff:np.float64(0.002889484982006252) - rollout_corr/log_ppl_diff_max:np.float64(0.026334106922149658) - rollout_corr/log_ppl_diff_min:np.float64(-0.00859677791595459) - rollout_corr/ppl_ratio:np.float64(1.0012316501233727) - rollout_corr/chi2_token:np.float64(0.002297403523698449) - rollout_corr/chi2_seq:np.float64(1.2100839058402926) - actor/pg_loss:np.float64(0.004832616308704019) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006622324299314641) - actor/ppo_kl:np.float64(0.00027824501646955824) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(57253.49309117114) - self_distillation/token_reweight_w_std:np.float64(780711.6470303052) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9960764090064913) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10869685839861631) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.5899917781352997) - perf/mfu/actor:np.float64(0.03340075709542584) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.18029022216797) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0030969788786023855 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0030969788786023855 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:266.609375 - response_length/max:611.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.609375 - response_length_non_aborted/max:611.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.40625 - prompt_length/max:709.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014692358672618866 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2525565940886736) - timing_s/agent_loop/generate_sequences/max:np.float64(4.980492563918233) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.909446031990228) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.980492563918233) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:406 - timing_s/agent_loop/slowest/response_length:611 - timing_s/gen:10.703484870493412 - timing_s/reward:0.056105511263012886 - timing_s/old_log_prob:2.315383156761527 - timing_s/adv:0.49405837804079056 - timing_s/update_actor:15.426462525501847 - timing_s/step:29.082023203372955 - timing_s/stop_profile:0.0001284964382648468 - timing_per_token_ms/adv:0.003427818791391159 - timing_per_token_ms/gen:0.156823021603666 - timing_per_token_ms/update_actor:0.10703010105668309 - perf/total_num_tokens:144132 - perf/time_per_step:29.082023203372955 - perf/throughput:619.5064172120746 (TaskRunner pid=2026458) Training Progress: 26%|██▌ | 26/100 [19:46<47:51, 38.80s/it] (TaskRunner pid=2026458) step:27 - global_seqlen/min:14932 - global_seqlen/max:19899 - global_seqlen/minmax_diff:4967 - global_seqlen/balanced_min:17014 - global_seqlen/balanced_max:17065 - global_seqlen/mean:17022.125 - actor/entropy:0.27002352476119995 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2583366632461548 - training/rollout_probs_diff_mean:0.005672288592904806 - training/rollout_probs_diff_std:0.013904700987040997 - training/rollout_actor_probs_pearson_corr:0.9982785582542419 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76171875 - self_distillation/correct_sample_fraction:0.5 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76171875 - rollout_corr/rollout_is_mean:1.0000253915786743 - rollout_corr/rollout_is_ratio_fraction_high:5.697275264537893e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4243188161344733e-05 - rollout_corr/rollout_is_max:2.935737371444702 - rollout_corr/rollout_is_min:0.4851767420768738 - rollout_corr/rollout_is_std:0.04337328299880028 - rollout_corr/rollout_is_eff_sample_size:0.9981222906269569 - rollout_corr/rollout_is_seq_mean:0.9999536275863647 - rollout_corr/rollout_is_seq_std:0.002799020381644368 - rollout_corr/rollout_is_seq_max:1.0089412927627563 - rollout_corr/rollout_is_seq_min:0.991137683391571 - rollout_corr/rollout_is_seq_max_deviation:0.008941292762756348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3306819731369615) - rollout_corr/training_log_ppl:np.float64(0.28223902196623385) - rollout_corr/kl:np.float64(0.0010320211754617503) - rollout_corr/k3_kl:np.float64(0.0012100190330102123) - rollout_corr/rollout_ppl:np.float64(1.3292772802524269) - rollout_corr/rollout_log_ppl:np.float64(0.28120700089493766) - rollout_corr/log_ppl_diff:np.float64(0.0010320210712961853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025203333934769034) - rollout_corr/log_ppl_diff_max:np.float64(0.011266171932220459) - rollout_corr/log_ppl_diff_min:np.float64(-0.00728076696395874) - rollout_corr/ppl_ratio:np.float64(1.0010370928794146) - rollout_corr/chi2_token:np.float64(0.002755629364401102) - rollout_corr/chi2_seq:np.float64(0.8209603114519268) - actor/pg_loss:np.float64(0.0054146344773471355) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001441872691430035) - actor/ppo_kl:np.float64(-3.021463783170475e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76171875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76171875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5) - self_distillation/token_reweight_w_mean:np.float64(149585.24241724727) - self_distillation/token_reweight_w_std:np.float64(2245182.637069527) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022315112873912) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10743028059368953) - self_distillation/empty_target_batch:np.float64(0.23828125) - actor/grad_norm:np.float64(0.5845905467867851) - perf/mfu/actor:np.float64(0.031846668272466384) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.01116561889648) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:0 - critic/score/mean:0.5 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0008261048933491111 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0008261048933491111 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.25390625 - response_length/max:877.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.25390625 - response_length_non_aborted/max:877.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.6875 - prompt_length/max:380.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001650247722864151 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4051890634000301) - timing_s/agent_loop/generate_sequences/max:np.float64(6.235670423135161) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9710693403030746) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.235670423135161) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:275 - timing_s/agent_loop/slowest/response_length:877 - timing_s/gen:12.108512347564101 - timing_s/reward:0.05487024039030075 - timing_s/old_log_prob:2.2881970033049583 - timing_s/adv:0.46778740733861923 - timing_s/update_actor:15.23558795452118 - timing_s/step:30.242075925692916 - timing_s/stop_profile:0.0001263841986656189 - timing_per_token_ms/adv:0.003435142552256396 - timing_per_token_ms/gen:0.17246382013081088 - timing_per_token_ms/update_actor:0.11188077248376142 - perf/total_num_tokens:136177 - perf/time_per_step:30.242075925692916 - perf/throughput:562.8623194328543 (TaskRunner pid=2026458) Training Progress: 27%|██▋ | 27/100 [20:16<44:06, 36.25s/it] (TaskRunner pid=2026458) step:28 - global_seqlen/min:15064 - global_seqlen/max:18912 - global_seqlen/minmax_diff:3848 - global_seqlen/balanced_min:17172 - global_seqlen/balanced_max:17302 - global_seqlen/mean:17191.25 - actor/entropy:0.26276373863220215 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491259455680847 - training/rollout_probs_diff_mean:0.005732021760195494 - training/rollout_probs_diff_std:0.014186778105795383 - training/rollout_actor_probs_pearson_corr:0.9981420636177063 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999222159385681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4750567970622797e-05 - rollout_corr/rollout_is_max:1.8682423830032349 - rollout_corr/rollout_is_min:0.4175392985343933 - rollout_corr/rollout_is_std:0.043118301779031754 - rollout_corr/rollout_is_eff_sample_size:0.9981441433187315 - rollout_corr/rollout_is_seq_mean:0.9998924732208252 - rollout_corr/rollout_is_seq_std:0.0026770406402647495 - rollout_corr/rollout_is_seq_max:1.0065722465515137 - rollout_corr/rollout_is_seq_min:0.9909303188323975 - rollout_corr/rollout_is_seq_max_deviation:0.009069681167602539 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3176847058348358) - rollout_corr/training_log_ppl:np.float64(0.2727040337776998) - rollout_corr/kl:np.float64(0.0012852028201848498) - rollout_corr/k3_kl:np.float64(0.001238845826378565) - rollout_corr/rollout_ppl:np.float64(1.315975757315755) - rollout_corr/rollout_log_ppl:np.float64(0.27141882907017134) - rollout_corr/log_ppl_diff:np.float64(0.0012852047075284645) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026650355284800753) - rollout_corr/log_ppl_diff_max:np.float64(0.011441320180892944) - rollout_corr/log_ppl_diff_min:np.float64(-0.011876940727233887) - rollout_corr/ppl_ratio:np.float64(1.0012910023797303) - rollout_corr/chi2_token:np.float64(0.002501809736713767) - rollout_corr/chi2_seq:np.float64(1.4115001251921058) - actor/pg_loss:np.float64(0.005721935303881764) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001080054133581143) - actor/ppo_kl:np.float64(0.00020560735394870022) - actor/pg_clipfrac_lower:np.float64(3.0048076951061375e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(263679.60727620986) - self_distillation/token_reweight_w_std:np.float64(3103030.912776855) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007451251149178) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10346215419122018) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.6124091595411301) - perf/mfu/actor:np.float64(0.03192732787611963) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.95586776733398) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00606801500543952 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00606801500543952 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:264.8203125 - response_length/max:1012.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:264.8203125 - response_length_non_aborted/max:1012.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.40625 - prompt_length/max:443.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001549404114484787 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3163175880908966) - timing_s/agent_loop/generate_sequences/max:np.float64(6.851215856149793) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.895652609608078) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.851215856149793) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:255 - timing_s/agent_loop/slowest/response_length:1012 - timing_s/gen:12.757756907492876 - timing_s/reward:0.05424255505204201 - timing_s/old_log_prob:2.304752092808485 - timing_s/adv:0.5271155349910259 - timing_s/update_actor:15.178545750677586 - timing_s/step:30.909547282382846 - timing_s/stop_profile:0.00012030825018882751 - timing_per_token_ms/adv:0.0038327312949249324 - timing_per_token_ms/gen:0.18818415947565972 - timing_per_token_ms/update_actor:0.11036534392988864 - perf/total_num_tokens:137530 - perf/time_per_step:30.909547282382846 - perf/throughput:556.179288002652 (TaskRunner pid=2026458) Training Progress: 28%|██▊ | 28/100 [20:47<41:35, 34.66s/it] (TaskRunner pid=2026458) step:29 - global_seqlen/min:14445 - global_seqlen/max:22059 - global_seqlen/minmax_diff:7614 - global_seqlen/balanced_min:19100 - global_seqlen/balanced_max:20098 - global_seqlen/mean:19231.25 - actor/entropy:0.2399301379919052 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5590871572494507 - training/rollout_probs_diff_mean:0.005236997734755278 - training/rollout_probs_diff_std:0.014018451794981956 - training/rollout_actor_probs_pearson_corr:0.9981542229652405 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.34375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:1.0000776052474976 - rollout_corr/rollout_is_ratio_fraction_high:2.4095805201795883e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.819161040359177e-05 - rollout_corr/rollout_is_max:3.816781997680664 - rollout_corr/rollout_is_min:0.17573890089988708 - rollout_corr/rollout_is_std:0.04291044920682907 - rollout_corr/rollout_is_eff_sample_size:0.9981623150356519 - rollout_corr/rollout_is_seq_mean:0.9999245405197144 - rollout_corr/rollout_is_seq_std:0.0027032301295548677 - rollout_corr/rollout_is_seq_max:1.008720874786377 - rollout_corr/rollout_is_seq_min:0.992788553237915 - rollout_corr/rollout_is_seq_max_deviation:0.008720874786376953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3173197288997471) - rollout_corr/training_log_ppl:np.float64(0.27068548381794244) - rollout_corr/kl:np.float64(0.0009471329129642214) - rollout_corr/k3_kl:np.float64(0.001118057121630045) - rollout_corr/rollout_ppl:np.float64(1.3160036713816226) - rollout_corr/rollout_log_ppl:np.float64(0.2697383483755402) - rollout_corr/log_ppl_diff:np.float64(0.0009471354424022138) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024268297711387277) - rollout_corr/log_ppl_diff_max:np.float64(0.010676145553588867) - rollout_corr/log_ppl_diff_min:np.float64(-0.008322864770889282) - rollout_corr/ppl_ratio:np.float64(1.0009521523024887) - rollout_corr/chi2_token:np.float64(0.0025950868148356676) - rollout_corr/chi2_seq:np.float64(1.0168882303405553) - actor/pg_loss:np.float64(0.004738333867862821) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009898098714984371) - actor/ppo_kl:np.float64(-0.0001235502958607526) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.34375) - self_distillation/token_reweight_w_mean:np.float64(148981.53039407637) - self_distillation/token_reweight_w_std:np.float64(2396815.2922347095) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019751288928092) - self_distillation/token_reweight_w_clip_frac:np.float64(0.088682441302808) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.5395381674170494) - perf/mfu/actor:np.float64(0.0350041185371077) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9663429260254) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:0 - critic/score/mean:0.34375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.34375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014469530433416367 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014469530433416367 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:324.2265625 - response_length/max:2192.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:324.2265625 - response_length_non_aborted/max:2192.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.75 - prompt_length/max:492.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000128256157040596 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1863264609128237) - timing_s/agent_loop/generate_sequences/max:np.float64(12.72119532711804) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.387789061780495) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.72119532711804) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:2192 - timing_s/gen:18.591661918908358 - timing_s/reward:0.05764943175017834 - timing_s/old_log_prob:2.3708045054227114 - timing_s/adv:0.6288922913372517 - timing_s/update_actor:15.632321024313569 - timing_s/step:37.36768543533981 - timing_s/stop_profile:0.00012124888598918915 - timing_per_token_ms/adv:0.004087697701249605 - timing_per_token_ms/gen:0.2239905293716821 - timing_per_token_ms/update_actor:0.10160754646937646 - perf/total_num_tokens:153850 - perf/time_per_step:37.36768543533981 - perf/throughput:514.6492156512428 (TaskRunner pid=2026458) Training Progress: 29%|██▉ | 29/100 [21:24<41:59, 35.49s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant for the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the options provided. The product contains a key fragment: "Cc2ccc(Cl)cc2C(F)(F)F", which is a benzene ring with a chlorine substituent and a trifluoromethyl group. This fragment is likely derived from a reactant that contains a similar structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A includes a structure with a benzene ring, a chlorine substituent, and a carboxylic acid group, but it does not contain the trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a structure with a benzene ring and a chlorine substituent, but it lacks the trifluoromethyl group and has a different functional group arrangement. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a structure with a benzene ring, a chlorine substituent, and a trifluoromethyl group, and it matches the required fragment in the product. It also includes a carboxylic acid group, which is consistent with the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a structure with a benzene ring and a trifluoromethyl group, but it lacks the chlorine substituent and has a different functional group arrangement. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, Option C is the only correct reactant that matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) C (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 0.0 (TaskRunner pid=2026458) [acc] 0.0 (TaskRunner pid=2026458) [pred] C (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2026458) step:30 - global_seqlen/min:15398 - global_seqlen/max:27317 - global_seqlen/minmax_diff:11919 - global_seqlen/balanced_min:18528 - global_seqlen/balanced_max:25696 - global_seqlen/mean:20322.875 - actor/entropy:0.21429221332073212 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29036805033683777 - training/rollout_probs_diff_mean:0.0045964838936924934 - training/rollout_probs_diff_std:0.012965062633156776 - training/rollout_actor_probs_pearson_corr:0.9982911944389343 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.41015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:0.9999114871025085 - rollout_corr/rollout_is_ratio_fraction_high:1.198825157189276e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.99412560404744e-05 - rollout_corr/rollout_is_max:2.0840299129486084 - rollout_corr/rollout_is_min:0.03206612169742584 - rollout_corr/rollout_is_std:0.04015055671334267 - rollout_corr/rollout_is_eff_sample_size:0.9983904085296447 - rollout_corr/rollout_is_seq_mean:0.9999212622642517 - rollout_corr/rollout_is_seq_std:0.003086200449615717 - rollout_corr/rollout_is_seq_max:1.0128618478775024 - rollout_corr/rollout_is_seq_min:0.9903483986854553 - rollout_corr/rollout_is_seq_max_deviation:0.012861847877502441 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3321797889657319) - rollout_corr/training_log_ppl:np.float64(0.28194430993880815) - rollout_corr/kl:np.float64(0.0011635422050204625) - rollout_corr/k3_kl:np.float64(0.0012278429346110897) - rollout_corr/rollout_ppl:np.float64(1.3305623242631555) - rollout_corr/rollout_log_ppl:np.float64(0.2807807675008007) - rollout_corr/log_ppl_diff:np.float64(0.0011635424380074255) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028164471932541346) - rollout_corr/log_ppl_diff_max:np.float64(0.01170569658279419) - rollout_corr/log_ppl_diff_min:np.float64(-0.010286390781402588) - rollout_corr/ppl_ratio:np.float64(1.0011705257929862) - rollout_corr/chi2_token:np.float64(0.0025684768334031105) - rollout_corr/chi2_seq:np.float64(0.56981481006369) - actor/pg_loss:np.float64(0.004291482153348625) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010325129969714908) - actor/ppo_kl:np.float64(2.040402626590776e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.41015625) - self_distillation/token_reweight_w_mean:np.float64(97715.03854049812) - self_distillation/token_reweight_w_std:np.float64(1362849.7468459418) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998477473855019) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08768687405972742) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.5216539651155472) - perf/mfu/actor:np.float64(0.03586380732978027) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.9196662902832) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/reward/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10296365255394938) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7833380952380953) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0895080232788665) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6183523809523809) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07343617415290264) - val-aux/sciknoweval/score/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/score/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10296365255394938) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7833380952380953) - val-aux/sciknoweval/score/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0895080232788665) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6183523809523809) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07343617415290264) - val-core/sciknoweval/acc/mean@16:np.float64(0.5943452380952381) - val-aux/sciknoweval/acc/std@16:np.float64(0.19385587331070453) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6712571428571429) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14416552478989664) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5181047619047618) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.18180031114620876) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.5949523809523809) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1935592702447506) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7259380952380953) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09597054475819111) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.4468809523809524) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.15668932421972914) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6092761904761904) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14936173723020238) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7630047619047619) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05768424104510231) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.3826380952380953) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.12589633211761853) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.615552380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10296365255394938) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7833380952380953) - val-core/sciknoweval/acc/best@16/std:np.float64(0.029000030584979297) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.32762380952380954) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0895080232788665) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6183523809523809) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07343617415290264) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:0 - critic/score/mean:0.41015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.41015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.08896331489086151 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.08896331489086151 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:325.83984375 - response_length/max:8192.0 - response_length/min:95.0 - response_length/clip_ratio:0.0078125 - response_length_non_aborted/mean:325.83984375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:95.0 - response_length_non_aborted/clip_ratio:0.0078125 - response/aborted_ratio:0.0 - prompt_length/mean:309.25 - prompt_length/max:516.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012153573334217072 - timing_s/agent_loop/generate_sequences/min:np.float64(0.691241292282939) - timing_s/agent_loop/generate_sequences/max:np.float64(49.47414601966739) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9101952763448935) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(49.47414601966739) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:218 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:55.4048219807446 - timing_s/reward:0.05735262483358383 - timing_s/old_log_prob:2.4137737192213535 - timing_s/adv:0.4771828204393387 - timing_s/update_actor:16.52942644432187 - timing_s/step:74.96756593696773 - timing_s/testing:65.94198310934007 - timing_s/save_checkpoint:6.872431704774499 - timing_s/stop_profile:9.614601731300354e-05 - timing_per_token_ms/adv:0.002935010551160568 - timing_per_token_ms/gen:0.6642069409667878 - timing_per_token_ms/update_actor:0.10166761865829681 - perf/total_num_tokens:162583 - perf/time_per_step:74.96756593696773 - perf/throughput:271.0889001930161 (TaskRunner pid=2026458) Training Progress: 30%|███ | 30/100 [23:52<1:20:43, 69.19s/it] (TaskRunner pid=2026458) step:31 - global_seqlen/min:15697 - global_seqlen/max:20497 - global_seqlen/minmax_diff:4800 - global_seqlen/balanced_min:18142 - global_seqlen/balanced_max:18440 - global_seqlen/mean:18179.75 - actor/entropy:0.24378487467765808 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4056262969970703 - training/rollout_probs_diff_mean:0.005305483937263489 - training/rollout_probs_diff_std:0.013869675807654858 - training/rollout_actor_probs_pearson_corr:0.9981986880302429 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0002444982528687 - rollout_corr/rollout_is_ratio_fraction_high:5.420200977823697e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.485351620242e-05 - rollout_corr/rollout_is_max:2.6529436111450195 - rollout_corr/rollout_is_min:0.2698281407356262 - rollout_corr/rollout_is_std:0.04314593970775604 - rollout_corr/rollout_is_eff_sample_size:0.9981427181140134 - rollout_corr/rollout_is_seq_mean:1.0003163814544678 - rollout_corr/rollout_is_seq_std:0.0029239451978355646 - rollout_corr/rollout_is_seq_max:1.0140212774276733 - rollout_corr/rollout_is_seq_min:0.9939373135566711 - rollout_corr/rollout_is_seq_max_deviation:0.01402127742767334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3019663002341986) - rollout_corr/training_log_ppl:np.float64(0.2597778052149806) - rollout_corr/kl:np.float64(0.001051263411028458) - rollout_corr/k3_kl:np.float64(0.0012008980110067569) - rollout_corr/rollout_ppl:np.float64(1.3005597270093858) - rollout_corr/rollout_log_ppl:np.float64(0.2587265418842435) - rollout_corr/log_ppl_diff:np.float64(0.0010512633307371289) - rollout_corr/log_ppl_abs_diff:np.float64(0.002555768267484382) - rollout_corr/log_ppl_diff_max:np.float64(0.010198384523391724) - rollout_corr/log_ppl_diff_min:np.float64(-0.010389238595962524) - rollout_corr/ppl_ratio:np.float64(1.00105693587102) - rollout_corr/chi2_token:np.float64(0.002714824862778187) - rollout_corr/chi2_seq:np.float64(1.5057584547903389) - actor/pg_loss:np.float64(0.004452571272850037) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007820382315912866) - actor/ppo_kl:np.float64(0.00035900160938950876) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(99177.65471042739) - self_distillation/token_reweight_w_std:np.float64(1529254.068720376) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9981596393045038) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09003465421847068) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.5145964473485947) - perf/mfu/actor:np.float64(0.0335625674385555) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.96116638183594) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0012466462794691324 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0012466462794691324 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:288.2734375 - response_length/max:769.0 - response_length/min:118.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:288.2734375 - response_length_non_aborted/max:769.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.84375 - prompt_length/max:635.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.449354648590088e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2310489993542433) - timing_s/agent_loop/generate_sequences/max:np.float64(5.724285513162613) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1068192758539226) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.724285513162613) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:635 - timing_s/agent_loop/slowest/response_length:769 - timing_s/gen:11.310221493244171 - timing_s/reward:0.053344203159213066 - timing_s/old_log_prob:2.4446758944541216 - timing_s/adv:0.550066627562046 - timing_s/update_actor:15.255971036851406 - timing_s/step:29.709052259102464 - timing_s/stop_profile:0.0001292899250984192 - timing_per_token_ms/adv:0.003782138282718726 - timing_per_token_ms/gen:0.15325918714930176 - timing_per_token_ms/update_actor:0.10489673288171872 - perf/total_num_tokens:145438 - perf/time_per_step:29.709052259102464 - perf/throughput:611.9262856804853 (TaskRunner pid=2026458) Training Progress: 31%|███ | 31/100 [24:22<1:05:57, 57.36s/it] (TaskRunner pid=2026458) step:32 - global_seqlen/min:13741 - global_seqlen/max:20093 - global_seqlen/minmax_diff:6352 - global_seqlen/balanced_min:17020 - global_seqlen/balanced_max:17022 - global_seqlen/mean:17021.5 - actor/entropy:0.2510213851928711 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3174631893634796 - training/rollout_probs_diff_mean:0.005626973696053028 - training/rollout_probs_diff_std:0.014290885999798775 - training/rollout_actor_probs_pearson_corr:0.9980513453483582 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:1.0003660917282104 - rollout_corr/rollout_is_ratio_fraction_high:4.35514775745105e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.35514775745105e-05 - rollout_corr/rollout_is_max:2.588045358657837 - rollout_corr/rollout_is_min:0.49156951904296875 - rollout_corr/rollout_is_std:0.04363907128572464 - rollout_corr/rollout_is_eff_sample_size:0.9981006764057321 - rollout_corr/rollout_is_seq_mean:1.0004558563232422 - rollout_corr/rollout_is_seq_std:0.0026327052619308233 - rollout_corr/rollout_is_seq_max:1.0082557201385498 - rollout_corr/rollout_is_seq_min:0.9930630326271057 - rollout_corr/rollout_is_seq_max_deviation:0.008255720138549805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3111304240301251) - rollout_corr/training_log_ppl:np.float64(0.2656050497171236) - rollout_corr/kl:np.float64(0.0011632751664905072) - rollout_corr/k3_kl:np.float64(0.0012394887605751137) - rollout_corr/rollout_ppl:np.float64(1.3095909063704312) - rollout_corr/rollout_log_ppl:np.float64(0.2644417754927417) - rollout_corr/log_ppl_diff:np.float64(0.0011632742243818939) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027760078373830765) - rollout_corr/log_ppl_diff_max:np.float64(0.027847379446029663) - rollout_corr/log_ppl_diff_min:np.float64(-0.00688682496547699) - rollout_corr/ppl_ratio:np.float64(1.0011707202065736) - rollout_corr/chi2_token:np.float64(0.00264995195902884) - rollout_corr/chi2_seq:np.float64(1.4071137343998998) - actor/pg_loss:np.float64(0.003960146103054285) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013795204131383798) - actor/ppo_kl:np.float64(0.0005964390661219454) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(250012.3793359641) - self_distillation/token_reweight_w_std:np.float64(2860040.7311532954) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007816257420927) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07969121757196262) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.522880308330059) - perf/mfu/actor:np.float64(0.031496540810405246) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.58367156982422) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:0 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019834069535136223 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019834069535136223 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:269.078125 - response_length/max:628.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.078125 - response_length_non_aborted/max:628.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.84375 - prompt_length/max:404.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002002175897359848 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3522796910256147) - timing_s/agent_loop/generate_sequences/max:np.float64(4.971057282760739) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.862827809440205) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.971057282760739) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:286 - timing_s/agent_loop/slowest/response_length:628 - timing_s/gen:10.714683732017875 - timing_s/reward:0.05403785966336727 - timing_s/old_log_prob:2.382686661556363 - timing_s/adv:0.4786151312291622 - timing_s/update_actor:15.277625290676951 - timing_s/step:28.9955370221287 - timing_s/stop_profile:0.00011957064270973206 - timing_per_token_ms/adv:0.003514783738427593 - timing_per_token_ms/gen:0.1555467703968683 - timing_per_token_ms/update_actor:0.1121935881875639 - perf/total_num_tokens:136172 - perf/time_per_step:28.9955370221287 - perf/throughput:587.0386186332606 (TaskRunner pid=2026458) Training Progress: 32%|███▏ | 32/100 [24:51<55:22, 48.86s/it] (TaskRunner pid=2026458) step:33 - global_seqlen/min:15641 - global_seqlen/max:20620 - global_seqlen/minmax_diff:4979 - global_seqlen/balanced_min:18180 - global_seqlen/balanced_max:18182 - global_seqlen/mean:18181.0 - actor/entropy:0.28735455870628357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9959163069725037 - training/rollout_probs_diff_mean:0.00580747751519084 - training/rollout_probs_diff_std:0.015400327742099762 - training/rollout_actor_probs_pearson_corr:0.9980352520942688 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9997233152389526 - rollout_corr/rollout_is_ratio_fraction_high:1.423528101440752e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014235280104912817 - rollout_corr/rollout_is_max:2.051370859146118 - rollout_corr/rollout_is_min:0.0007158105727285147 - rollout_corr/rollout_is_std:0.04281798005104065 - rollout_corr/rollout_is_eff_sample_size:0.998169085060141 - rollout_corr/rollout_is_seq_mean:0.9997956156730652 - rollout_corr/rollout_is_seq_std:0.003073018742725253 - rollout_corr/rollout_is_seq_max:1.0114701986312866 - rollout_corr/rollout_is_seq_min:0.9903202056884766 - rollout_corr/rollout_is_seq_max_deviation:0.011470198631286621 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3598110149614513) - rollout_corr/training_log_ppl:np.float64(0.30384603922721) - rollout_corr/kl:np.float64(0.0014454446327292203) - rollout_corr/k3_kl:np.float64(0.001423622587481077) - rollout_corr/rollout_ppl:np.float64(1.3577961758710444) - rollout_corr/rollout_log_ppl:np.float64(0.3024005940533243) - rollout_corr/log_ppl_diff:np.float64(0.0014454451738856733) - rollout_corr/log_ppl_abs_diff:np.float64(0.002951021713670343) - rollout_corr/log_ppl_diff_max:np.float64(0.02602332830429077) - rollout_corr/log_ppl_diff_min:np.float64(-0.010346651077270508) - rollout_corr/ppl_ratio:np.float64(1.001454961951822) - rollout_corr/chi2_token:np.float64(0.002324617700651288) - rollout_corr/chi2_seq:np.float64(0.6557224565185606) - actor/pg_loss:np.float64(0.0037985126255080104) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009114227987083723) - actor/ppo_kl:np.float64(7.80257138011109e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(186856.52135437424) - self_distillation/token_reweight_w_std:np.float64(2469605.76739274) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999388714786619) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09890055964933708) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5402719005942345) - perf/mfu/actor:np.float64(0.03298787544887611) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.31770038604736) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003960967063903809 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003960967063903809 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.40625 - response_length/max:556.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.40625 - response_length_non_aborted/max:556.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.75 - prompt_length/max:474.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001493934541940689 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3925712388008833) - timing_s/agent_loop/generate_sequences/max:np.float64(4.813172671943903) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.01079661748372) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.813172671943903) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:198 - timing_s/agent_loop/slowest/response_length:556 - timing_s/gen:10.570713378489017 - timing_s/reward:0.056401949375867844 - timing_s/old_log_prob:2.26583200879395 - timing_s/adv:0.4962475970387459 - timing_s/update_actor:15.83989292383194 - timing_s/step:29.316704677417874 - timing_s/stop_profile:3.923475742340088e-05 - timing_per_token_ms/adv:0.00341185576315072 - timing_per_token_ms/gen:0.1504770723506579 - timing_per_token_ms/update_actor:0.10890416453874883 - perf/total_num_tokens:145448 - perf/time_per_step:29.316704677417874 - perf/throughput:620.1583772818947 (TaskRunner pid=2026458) Training Progress: 33%|███▎ | 33/100 [25:20<48:01, 43.01s/it] (TaskRunner pid=2026458) step:34 - global_seqlen/min:15048 - global_seqlen/max:22716 - global_seqlen/minmax_diff:7668 - global_seqlen/balanced_min:18406 - global_seqlen/balanced_max:18407 - global_seqlen/mean:18406.75 - actor/entropy:0.2822253406047821 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3892279863357544 - training/rollout_probs_diff_mean:0.00599341094493866 - training/rollout_probs_diff_std:0.014615834690630436 - training/rollout_actor_probs_pearson_corr:0.9981156587600708 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9999809265136719 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00012492018868215382 - rollout_corr/rollout_is_max:1.9517436027526855 - rollout_corr/rollout_is_min:0.10854519158601761 - rollout_corr/rollout_is_std:0.04540359228849411 - rollout_corr/rollout_is_eff_sample_size:0.9979426360193112 - rollout_corr/rollout_is_seq_mean:0.9999065399169922 - rollout_corr/rollout_is_seq_std:0.003091080579906702 - rollout_corr/rollout_is_seq_max:1.0100407600402832 - rollout_corr/rollout_is_seq_min:0.9879905581474304 - rollout_corr/rollout_is_seq_max_deviation:0.01200944185256958 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3448055190965533) - rollout_corr/training_log_ppl:np.float64(0.29254834007588215) - rollout_corr/kl:np.float64(0.001122740535322464) - rollout_corr/k3_kl:np.float64(0.0011786174198959998) - rollout_corr/rollout_ppl:np.float64(1.3432867326773703) - rollout_corr/rollout_log_ppl:np.float64(0.2914255977375433) - rollout_corr/log_ppl_diff:np.float64(0.0011227423383388668) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025718869001138955) - rollout_corr/log_ppl_diff_max:np.float64(0.014177501201629639) - rollout_corr/log_ppl_diff_min:np.float64(-0.006468132138252258) - rollout_corr/ppl_ratio:np.float64(1.0011283229105175) - rollout_corr/chi2_token:np.float64(0.002466450445353985) - rollout_corr/chi2_seq:np.float64(0.9479083714541048) - actor/pg_loss:np.float64(0.005841488018631935) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011660458308142552) - actor/ppo_kl:np.float64(-8.996075499190237e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(205396.02536904486) - self_distillation/token_reweight_w_std:np.float64(2610298.4065771694) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991585344541818) - self_distillation/token_reweight_w_clip_frac:np.float64(0.12581025002873503) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.6120489835739136) - perf/mfu/actor:np.float64(0.03423697654860308) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.21736526489258) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012471199035644531 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012471199035644531 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:281.4296875 - response_length/max:743.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.4296875 - response_length_non_aborted/max:743.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.78125 - prompt_length/max:621.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.58306884765625e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4199188239872456) - timing_s/agent_loop/generate_sequences/max:np.float64(5.984945947304368) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2054700024527847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.984945947304368) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:207 - timing_s/agent_loop/slowest/response_length:743 - timing_s/gen:11.809425836429 - timing_s/reward:0.05546233803033829 - timing_s/old_log_prob:2.252874342724681 - timing_s/adv:0.49194580502808094 - timing_s/update_actor:15.391067944467068 - timing_s/step:30.08788885548711 - timing_s/stop_profile:0.00011718645691871643 - timing_per_token_ms/adv:0.003340797567659153 - timing_per_token_ms/gen:0.16391507976055575 - timing_per_token_ms/update_actor:0.10452054235855779 - perf/total_num_tokens:147254 - perf/time_per_step:30.08788885548711 - perf/throughput:611.7660859626305 (TaskRunner pid=2026458) Training Progress: 34%|███▍ | 34/100 [25:50<43:03, 39.14s/it] (TaskRunner pid=2026458) step:35 - global_seqlen/min:14987 - global_seqlen/max:19906 - global_seqlen/minmax_diff:4919 - global_seqlen/balanced_min:16938 - global_seqlen/balanced_max:16939 - global_seqlen/mean:16938.25 - actor/entropy:0.294697642326355 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.22079068422317505 - training/rollout_probs_diff_mean:0.005864571314305067 - training/rollout_probs_diff_std:0.013605127111077309 - training/rollout_actor_probs_pearson_corr:0.9984296560287476 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.5859375 - self_distillation/correct_sample_fraction:0.5078125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.5859375 - rollout_corr/rollout_is_mean:1.00005042552948 - rollout_corr/rollout_is_ratio_fraction_high:3.1109037081478164e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.1109037081478164e-05 - rollout_corr/rollout_is_max:2.7191102504730225 - rollout_corr/rollout_is_min:0.37482571601867676 - rollout_corr/rollout_is_std:0.04332791268825531 - rollout_corr/rollout_is_eff_sample_size:0.9981263285522803 - rollout_corr/rollout_is_seq_mean:1.0001163482666016 - rollout_corr/rollout_is_seq_std:0.0030649646650999784 - rollout_corr/rollout_is_seq_max:1.0084450244903564 - rollout_corr/rollout_is_seq_min:0.989280641078949 - rollout_corr/rollout_is_seq_max_deviation:0.010719358921051025 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3609930002130568) - rollout_corr/training_log_ppl:np.float64(0.3051015918608755) - rollout_corr/kl:np.float64(0.0011746971918284999) - rollout_corr/k3_kl:np.float64(0.0011032617354658214) - rollout_corr/rollout_ppl:np.float64(1.3593582352623343) - rollout_corr/rollout_log_ppl:np.float64(0.30392689420841634) - rollout_corr/log_ppl_diff:np.float64(0.0011746976524591446) - rollout_corr/log_ppl_abs_diff:np.float64(0.002850949880667031) - rollout_corr/log_ppl_diff_max:np.float64(0.012322753667831421) - rollout_corr/log_ppl_diff_min:np.float64(-0.010998740792274475) - rollout_corr/ppl_ratio:np.float64(1.0011811451986432) - rollout_corr/chi2_token:np.float64(0.0021009566262364388) - rollout_corr/chi2_seq:np.float64(1.5435644236858934) - actor/pg_loss:np.float64(0.0014815301401540637) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00020538957323879004) - actor/ppo_kl:np.float64(0.00030627488916046275) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.5859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5078125) - self_distillation/token_reweight_w_mean:np.float64(94700.63521376462) - self_distillation/token_reweight_w_std:np.float64(1075744.331360741) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004025450907648) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03615615499438718) - self_distillation/empty_target_batch:np.float64(0.4140625) - actor/grad_norm:np.float64(0.3742603175342083) - perf/mfu/actor:np.float64(0.031170659142369998) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1671371459961) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:0 - critic/score/mean:0.5078125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5078125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029106393922120333 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0029106393922120333 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:251.1328125 - response_length/max:502.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:251.1328125 - response_length_non_aborted/max:502.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.1875 - prompt_length/max:514.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001547783613204956 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1277420744299889) - timing_s/agent_loop/generate_sequences/max:np.float64(4.495345920324326) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.789818469223974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.495345920324326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:355 - timing_s/agent_loop/slowest/response_length:502 - timing_s/gen:10.292290568351746 - timing_s/reward:0.054091084748506546 - timing_s/old_log_prob:2.2543541602790356 - timing_s/adv:0.4955630674958229 - timing_s/update_actor:15.42586961016059 - timing_s/step:28.60832902789116 - timing_s/stop_profile:3.491714596748352e-05 - timing_per_token_ms/adv:0.003657130071700315 - timing_per_token_ms/gen:0.16009162495491905 - timing_per_token_ms/update_actor:0.11383901532153992 - perf/total_num_tokens:135506 - perf/time_per_step:28.60832902789116 - perf/throughput:592.0740768706333 (TaskRunner pid=2026458) Training Progress: 35%|███▌ | 35/100 [26:19<38:59, 35.99s/it] (TaskRunner pid=2026458) step:36 - global_seqlen/min:14431 - global_seqlen/max:20659 - global_seqlen/minmax_diff:6228 - global_seqlen/balanced_min:17408 - global_seqlen/balanced_max:17413 - global_seqlen/mean:17411.75 - actor/entropy:0.2818928360939026 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30500367283821106 - training/rollout_probs_diff_mean:0.005986991338431835 - training/rollout_probs_diff_std:0.014329608529806137 - training/rollout_actor_probs_pearson_corr:0.9982293248176575 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0000543594360352 - rollout_corr/rollout_is_ratio_fraction_high:1.4678683328384068e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.871473331353627e-05 - rollout_corr/rollout_is_max:3.0542242527008057 - rollout_corr/rollout_is_min:0.18286029994487762 - rollout_corr/rollout_is_std:0.044746387749910355 - rollout_corr/rollout_is_eff_sample_size:0.9980018804571298 - rollout_corr/rollout_is_seq_mean:1.0000989437103271 - rollout_corr/rollout_is_seq_std:0.0029557279776781797 - rollout_corr/rollout_is_seq_max:1.0128530263900757 - rollout_corr/rollout_is_seq_min:0.992638111114502 - rollout_corr/rollout_is_seq_max_deviation:0.012853026390075684 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3384348950348794) - rollout_corr/training_log_ppl:np.float64(0.28769120943616144) - rollout_corr/kl:np.float64(0.0010638975770804748) - rollout_corr/k3_kl:np.float64(0.0012365762453327989) - rollout_corr/rollout_ppl:np.float64(1.3369859019294381) - rollout_corr/rollout_log_ppl:np.float64(0.28662730925134383) - rollout_corr/log_ppl_diff:np.float64(0.0010639001848176122) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025838720030151308) - rollout_corr/log_ppl_diff_max:np.float64(0.01416441798210144) - rollout_corr/log_ppl_diff_min:np.float64(-0.009993910789489746) - rollout_corr/ppl_ratio:np.float64(1.0010696118697524) - rollout_corr/chi2_token:np.float64(0.002987496554851532) - rollout_corr/chi2_seq:np.float64(0.8903114418499172) - actor/pg_loss:np.float64(0.0051360129145905375) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000932457276576315) - actor/ppo_kl:np.float64(0.00010139281163645819) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(95758.95235729823) - self_distillation/token_reweight_w_std:np.float64(1392223.179759777) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9973206941504031) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10905869025737047) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.5497298240661621) - perf/mfu/actor:np.float64(0.03256102036135432) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.1156120300293) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.007596218958497047 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.007596218958497047 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:266.1171875 - response_length/max:662.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.1171875 - response_length_non_aborted/max:662.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.0 - prompt_length/max:704.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.2092596888542175e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3356867656111717) - timing_s/agent_loop/generate_sequences/max:np.float64(5.081710338592529) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.919884198134241) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.081710338592529) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:379 - timing_s/agent_loop/slowest/response_length:662 - timing_s/gen:10.837040143087506 - timing_s/reward:0.05424739792943001 - timing_s/old_log_prob:2.3121078684926033 - timing_s/adv:0.5094937365502119 - timing_s/update_actor:15.35438490100205 - timing_s/step:29.153722267597914 - timing_s/stop_profile:3.104284405708313e-05 - timing_per_token_ms/adv:0.0036576861641579098 - timing_per_token_ms/gen:0.1590734835905162 - timing_per_token_ms/update_actor:0.11023005227075143 - perf/total_num_tokens:139294 - perf/time_per_step:29.153722267597914 - perf/throughput:597.2393452945732 (TaskRunner pid=2026458) Training Progress: 36%|███▌ | 36/100 [26:48<36:12, 33.94s/it] (TaskRunner pid=2026458) step:37 - global_seqlen/min:15833 - global_seqlen/max:21178 - global_seqlen/minmax_diff:5345 - global_seqlen/balanced_min:18583 - global_seqlen/balanced_max:18587 - global_seqlen/mean:18585.625 - actor/entropy:0.285804808139801 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4652300179004669 - training/rollout_probs_diff_mean:0.005965293850749731 - training/rollout_probs_diff_std:0.014233550056815147 - training/rollout_actor_probs_pearson_corr:0.9982559680938721 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:1.0002163648605347 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.9148995963623747e-05 - rollout_corr/rollout_is_max:1.695287823677063 - rollout_corr/rollout_is_min:0.16980920732021332 - rollout_corr/rollout_is_std:0.04377271980047226 - rollout_corr/rollout_is_eff_sample_size:0.9980885633504123 - rollout_corr/rollout_is_seq_mean:1.000288486480713 - rollout_corr/rollout_is_seq_std:0.0028211281169205904 - rollout_corr/rollout_is_seq_max:1.0087376832962036 - rollout_corr/rollout_is_seq_min:0.9919080138206482 - rollout_corr/rollout_is_seq_max_deviation:0.008737683296203613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3575056260451674) - rollout_corr/training_log_ppl:np.float64(0.3016296330606565) - rollout_corr/kl:np.float64(0.0007570147956474926) - rollout_corr/k3_kl:np.float64(0.0012164926273499077) - rollout_corr/rollout_ppl:np.float64(1.3564373790286481) - rollout_corr/rollout_log_ppl:np.float64(0.30087261548032984) - rollout_corr/log_ppl_diff:np.float64(0.0007570175803266466) - rollout_corr/log_ppl_abs_diff:np.float64(0.002560925087891519) - rollout_corr/log_ppl_diff_max:np.float64(0.011357903480529785) - rollout_corr/log_ppl_diff_min:np.float64(-0.008871495723724365) - rollout_corr/ppl_ratio:np.float64(1.000762677518651) - rollout_corr/chi2_token:np.float64(0.0033016859088093042) - rollout_corr/chi2_seq:np.float64(1.0105145464185625) - actor/pg_loss:np.float64(0.0031967939576134086) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006711441328661749) - actor/ppo_kl:np.float64(3.3348735911431504e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.515625) - self_distillation/token_reweight_w_mean:np.float64(214368.9940047427) - self_distillation/token_reweight_w_std:np.float64(2919033.6983316364) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0007499558851123) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07807663449784741) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.4402663931250572) - perf/mfu/actor:np.float64(0.034623907048585915) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.02531814575195) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0031590224243700504 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0031590224243700504 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:268.01953125 - response_length/max:686.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:268.01953125 - response_length_non_aborted/max:686.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.78125 - prompt_length/max:1009.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.923235297203064e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.457766680046916) - timing_s/agent_loop/generate_sequences/max:np.float64(5.311068296432495) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9142332015690044) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.311068296432495) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:258 - timing_s/agent_loop/slowest/response_length:686 - timing_s/gen:11.104413088411093 - timing_s/reward:0.05539845488965511 - timing_s/old_log_prob:2.2851091288030148 - timing_s/adv:0.5124162267893553 - timing_s/update_actor:15.311178967356682 - timing_s/step:29.35571401193738 - timing_s/stop_profile:0.00011339224874973297 - timing_per_token_ms/adv:0.003446320925374821 - timing_per_token_ms/gen:0.1618412412867983 - timing_per_token_ms/update_actor:0.10297729406030656 - perf/total_num_tokens:148685 - perf/time_per_step:29.35571401193738 - perf/throughput:633.1177975246056 (TaskRunner pid=2026458) Training Progress: 37%|███▋ | 37/100 [27:18<34:12, 32.58s/it] (TaskRunner pid=2026458) step:38 - global_seqlen/min:15292 - global_seqlen/max:19759 - global_seqlen/minmax_diff:4467 - global_seqlen/balanced_min:17036 - global_seqlen/balanced_max:17040 - global_seqlen/mean:17038.625 - actor/entropy:0.2831912040710449 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4492228925228119 - training/rollout_probs_diff_mean:0.006005707662552595 - training/rollout_probs_diff_std:0.014447803609073162 - training/rollout_actor_probs_pearson_corr:0.9981924891471863 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9998399019241333 - rollout_corr/rollout_is_ratio_fraction_high:2.9010312573518604e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.3515468860277906e-05 - rollout_corr/rollout_is_max:4.976550579071045 - rollout_corr/rollout_is_min:0.1723451316356659 - rollout_corr/rollout_is_std:0.045169975608587265 - rollout_corr/rollout_is_eff_sample_size:0.997963056153362 - rollout_corr/rollout_is_seq_mean:0.999801754951477 - rollout_corr/rollout_is_seq_std:0.0029568150639533997 - rollout_corr/rollout_is_seq_max:1.01389741897583 - rollout_corr/rollout_is_seq_min:0.9917296171188354 - rollout_corr/rollout_is_seq_max_deviation:0.013897418975830078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3477132506668568) - rollout_corr/training_log_ppl:np.float64(0.29560438107000664) - rollout_corr/kl:np.float64(0.0015795158470695014) - rollout_corr/k3_kl:np.float64(0.0011915537743334426) - rollout_corr/rollout_ppl:np.float64(1.3455825243145227) - rollout_corr/rollout_log_ppl:np.float64(0.29402486264007166) - rollout_corr/log_ppl_diff:np.float64(0.0015795184299349785) - rollout_corr/log_ppl_abs_diff:np.float64(0.002752322005107999) - rollout_corr/log_ppl_diff_max:np.float64(0.010404467582702637) - rollout_corr/log_ppl_diff_min:np.float64(-0.006978675723075867) - rollout_corr/ppl_ratio:np.float64(1.0015854325611144) - rollout_corr/chi2_token:np.float64(0.0016510849818587303) - rollout_corr/chi2_seq:np.float64(0.42648625490255654) - actor/pg_loss:np.float64(0.004350315313786268) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001112287538489909) - actor/ppo_kl:np.float64(0.0002975818272203945) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(277881.75766049954) - self_distillation/token_reweight_w_std:np.float64(3405953.521727615) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9997533587738872) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09852648590458557) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.5438188463449478) - perf/mfu/actor:np.float64(0.03193194428398073) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.85803985595703) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00237884558737278 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00237884558737278 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:269.30078125 - response_length/max:549.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.30078125 - response_length_non_aborted/max:549.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.15625 - prompt_length/max:500.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013971328735351562 - timing_s/agent_loop/generate_sequences/min:np.float64(1.483020644634962) - timing_s/agent_loop/generate_sequences/max:np.float64(4.714750545099378) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0552332281222334) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.714750545099378) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:549 - timing_s/gen:10.464355397969484 - timing_s/reward:0.05412936769425869 - timing_s/old_log_prob:2.2855528369545937 - timing_s/adv:0.4730773251503706 - timing_s/update_actor:15.16877150721848 - timing_s/step:28.533766953274608 - timing_s/stop_profile:0.00012415647506713867 - timing_per_token_ms/adv:0.003470624281231398 - timing_per_token_ms/gen:0.15178711358943855 - timing_per_token_ms/update_actor:0.11128224480568767 - perf/total_num_tokens:136309 - perf/time_per_step:28.533766953274608 - perf/throughput:597.1389977321099 (TaskRunner pid=2026458) Training Progress: 38%|███▊ | 38/100 [27:46<32:25, 31.38s/it] (TaskRunner pid=2026458) step:39 - global_seqlen/min:15017 - global_seqlen/max:21775 - global_seqlen/minmax_diff:6758 - global_seqlen/balanced_min:17748 - global_seqlen/balanced_max:17751 - global_seqlen/mean:17750.375 - actor/entropy:0.2934376001358032 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744115471839905 - training/rollout_probs_diff_mean:0.006076316349208355 - training/rollout_probs_diff_std:0.014462748542428017 - training/rollout_actor_probs_pearson_corr:0.9982326030731201 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.7734375 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7734375 - rollout_corr/rollout_is_mean:1.0001943111419678 - rollout_corr/rollout_is_ratio_fraction_high:2.8678357921307907e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.735671584261581e-05 - rollout_corr/rollout_is_max:2.3144755363464355 - rollout_corr/rollout_is_min:0.44314810633659363 - rollout_corr/rollout_is_std:0.04461565613746643 - rollout_corr/rollout_is_eff_sample_size:0.9980141101447896 - rollout_corr/rollout_is_seq_mean:1.0003681182861328 - rollout_corr/rollout_is_seq_std:0.0030234723817557096 - rollout_corr/rollout_is_seq_max:1.012935996055603 - rollout_corr/rollout_is_seq_min:0.991753876209259 - rollout_corr/rollout_is_seq_max_deviation:0.012935996055603027 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3561590211465955) - rollout_corr/training_log_ppl:np.float64(0.30050787163781933) - rollout_corr/kl:np.float64(0.0008042539968293738) - rollout_corr/k3_kl:np.float64(0.0011359995904740572) - rollout_corr/rollout_ppl:np.float64(1.355069079902023) - rollout_corr/rollout_log_ppl:np.float64(0.2997036176966503) - rollout_corr/log_ppl_diff:np.float64(0.0008042539411690086) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024447946052532643) - rollout_corr/log_ppl_diff_max:np.float64(0.010187238454818726) - rollout_corr/log_ppl_diff_min:np.float64(-0.010493099689483643) - rollout_corr/ppl_ratio:np.float64(1.0008093006908894) - rollout_corr/chi2_token:np.float64(0.0029845864046365023) - rollout_corr/chi2_seq:np.float64(0.9699735399335623) - actor/pg_loss:np.float64(0.003664600197225809) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006868128334645007) - actor/ppo_kl:np.float64(0.00012619273251779362) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(231536.99509470444) - self_distillation/token_reweight_w_std:np.float64(3103805.8869296443) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012708925642073) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08035396633204073) - self_distillation/empty_target_batch:np.float64(0.2265625) - actor/grad_norm:np.float64(0.45319733768701553) - perf/mfu/actor:np.float64(0.03298738163573484) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.00569915771484) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:0 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005079654045403004 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005079654045403004 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.41796875 - response_length/max:558.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.41796875 - response_length_non_aborted/max:558.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:282.28125 - prompt_length/max:452.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.198114275932312e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3789098169654608) - timing_s/agent_loop/generate_sequences/max:np.float64(4.869980722665787) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.020734860765515) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.869980722665787) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:365 - timing_s/agent_loop/slowest/response_length:558 - timing_s/gen:10.739610413089395 - timing_s/reward:0.054600564762949944 - timing_s/old_log_prob:2.247015619650483 - timing_s/adv:0.5021204594522715 - timing_s/update_actor:15.248395329341292 - timing_s/step:28.878361692652106 - timing_s/stop_profile:0.00011930800974369049 - timing_per_token_ms/adv:0.0035359848697018477 - timing_per_token_ms/gen:0.15399719544429077 - timing_per_token_ms/update_actor:0.10738079709119731 - perf/total_num_tokens:142003 - perf/time_per_step:28.878361692652106 - perf/throughput:614.6600416226678 (TaskRunner pid=2026458) Training Progress: 39%|███▉ | 39/100 [28:15<31:09, 30.65s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to analyze the structure of the product and compare it with the options provided. The product contains a nitrogen atom bonded to a carbon chain, with a substituted aromatic ring (Cc2ccc(Cl)cc2) and a trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. The nitrogen is bonded to a substituted aromatic ring with a chlorine atom and a trifluoromethyl group, matching the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it lacks the specific chlorine and trifluoromethyl groups. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it includes an oxygen group (Oc2) instead of the chlorine and trifluoromethyl groups. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a structure with a ketone group (O=C), an aromatic ring (c2ccccc2), and a nitrogen atom (N) connected to a carbon chain. However, the substituents on the nitrogen do not match the product's structure, as it includes a double bond (CC=C) and a different trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Based on the structural analysis, Option A is the only one that matches the product's structure, making it the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2026458) step:40 - global_seqlen/min:16589 - global_seqlen/max:23076 - global_seqlen/minmax_diff:6487 - global_seqlen/balanced_min:19038 - global_seqlen/balanced_max:19042 - global_seqlen/mean:19040.5 - actor/entropy:0.3083128035068512 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.48448076844215393 - training/rollout_probs_diff_mean:0.006092434749007225 - training/rollout_probs_diff_std:0.01408663671463728 - training/rollout_actor_probs_pearson_corr:0.9983260631561279 - self_distillation/success_group_fraction:0.625 - self_distillation/success_sample_fraction:0.62109375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.62109375 - rollout_corr/rollout_is_mean:1.0004887580871582 - rollout_corr/rollout_is_ratio_fraction_high:2.767170371953398e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.383585185976699e-05 - rollout_corr/rollout_is_max:2.5501346588134766 - rollout_corr/rollout_is_min:0.2322635054588318 - rollout_corr/rollout_is_std:0.044838204979896545 - rollout_corr/rollout_is_eff_sample_size:0.9979954688959803 - rollout_corr/rollout_is_seq_mean:1.0004291534423828 - rollout_corr/rollout_is_seq_std:0.00280501926317811 - rollout_corr/rollout_is_seq_max:1.0100523233413696 - rollout_corr/rollout_is_seq_min:0.9925862550735474 - rollout_corr/rollout_is_seq_max_deviation:0.010052323341369629 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3748237513937056) - rollout_corr/training_log_ppl:np.float64(0.3138403666380327) - rollout_corr/kl:np.float64(0.0008364543262393553) - rollout_corr/k3_kl:np.float64(0.0011391592736345046) - rollout_corr/rollout_ppl:np.float64(1.3736085868440568) - rollout_corr/rollout_log_ppl:np.float64(0.3130039106181357) - rollout_corr/log_ppl_diff:np.float64(0.0008364560198970139) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023778348113410175) - rollout_corr/log_ppl_diff_max:np.float64(0.009519308805465698) - rollout_corr/log_ppl_diff_min:np.float64(-0.0073702335357666016) - rollout_corr/ppl_ratio:np.float64(1.0008412639144808) - rollout_corr/chi2_token:np.float64(0.0029417627956718206) - rollout_corr/chi2_seq:np.float64(0.8329082692507654) - actor/pg_loss:np.float64(0.0021506232442334294) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00047293603620346403) - actor/ppo_kl:np.float64(0.0002316894797989022) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.62109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(45010.86430421425) - self_distillation/token_reweight_w_std:np.float64(529273.9801185789) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998293393291533) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05143398354994133) - self_distillation/empty_target_batch:np.float64(0.37890625) - actor/grad_norm:np.float64(0.38673160597682) - perf/mfu/actor:np.float64(0.03526754309862706) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(143.86711502075195) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/reward/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09064171674985216) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.7905285714285714) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07673490521638164) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6577380952380952) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06656845162631347) - val-aux/sciknoweval/score/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/score/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/score/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09064171674985216) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.7905285714285714) - val-aux/sciknoweval/score/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07673490521638164) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6577380952380952) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06656845162631347) - val-core/sciknoweval/acc/mean@16:np.float64(0.6357142857142857) - val-aux/sciknoweval/acc/std@16:np.float64(0.16134453805796828) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6995523809523809) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.11925658429648062) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5717714285714286) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15210733801155413) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6355761904761905) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16083633014996535) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7442000000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07791528897318264) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5117523809523808) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1301078901202422) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.647747619047619) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.12547152462394415) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7727952380952381) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04748400982093751) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45995714285714284) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10409181623779554) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6544619047619048) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09064171674985216) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.7905285714285714) - val-core/sciknoweval/acc/best@16/std:np.float64(0.026333374195131425) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41540476190476183) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07673490521638164) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6577380952380952) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06656845162631347) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999857142857144) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002604287412166728) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994380952380952) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015362291495737217) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997095238095238) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0011396681698700574) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9989476190476191) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.001975811782419187) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999333333333333) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0005594782149609339) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980619047619048) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023394041530661654) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999995238095238) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.00015050933932646775) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9969285714285715) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.002278634869993062) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034849049989134073 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034849049989134073 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:282.328125 - response_length/max:680.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:282.328125 - response_length_non_aborted/max:680.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.6875 - prompt_length/max:501.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018454715609550476 - timing_s/agent_loop/generate_sequences/min:np.float64(1.287922067567706) - timing_s/agent_loop/generate_sequences/max:np.float64(5.481446422636509) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0874510004068725) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.481446422636509) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:272 - timing_s/agent_loop/slowest/response_length:680 - timing_s/gen:11.202703416347504 - timing_s/reward:0.05725381709635258 - timing_s/old_log_prob:2.3498482443392277 - timing_s/adv:0.48736315965652466 - timing_s/update_actor:15.279724219813943 - timing_s/step:29.463446816429496 - timing_s/testing:112.37399195507169 - timing_s/save_checkpoint:6.7144703436642885 - timing_s/stop_profile:0.00011608190834522247 - timing_per_token_ms/adv:0.003199516554558209 - timing_per_token_ms/gen:0.15499894039995993 - timing_per_token_ms/update_actor:0.10031068130966848 - perf/total_num_tokens:152324 - perf/time_per_step:29.463446816429496 - perf/throughput:646.2414298853378 (TaskRunner pid=2026458) Training Progress: 40%|████ | 40/100 [30:44<1:06:02, 66.04s/it] (TaskRunner pid=2026458) step:41 - global_seqlen/min:14161 - global_seqlen/max:22430 - global_seqlen/minmax_diff:8269 - global_seqlen/balanced_min:17529 - global_seqlen/balanced_max:17569 - global_seqlen/mean:17540.0 - actor/entropy:0.28421929478645325 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3167548179626465 - training/rollout_probs_diff_mean:0.005681131035089493 - training/rollout_probs_diff_std:0.013473561964929104 - training/rollout_actor_probs_pearson_corr:0.9984502792358398 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0002758502960205 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.967482328414917 - rollout_corr/rollout_is_min:0.5226783752441406 - rollout_corr/rollout_is_std:0.04252113774418831 - rollout_corr/rollout_is_eff_sample_size:0.998196166076475 - rollout_corr/rollout_is_seq_mean:1.000309944152832 - rollout_corr/rollout_is_seq_std:0.002816120395436883 - rollout_corr/rollout_is_seq_max:1.0100390911102295 - rollout_corr/rollout_is_seq_min:0.9916365146636963 - rollout_corr/rollout_is_seq_max_deviation:0.010039091110229492 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3460121774114668) - rollout_corr/training_log_ppl:np.float64(0.29434393605333753) - rollout_corr/kl:np.float64(0.0006344841518632904) - rollout_corr/k3_kl:np.float64(0.0010341189710061371) - rollout_corr/rollout_ppl:np.float64(1.3451492232270539) - rollout_corr/rollout_log_ppl:np.float64(0.2937094499648083) - rollout_corr/log_ppl_diff:np.float64(0.0006344860885292292) - rollout_corr/log_ppl_abs_diff:np.float64(0.002157465787604451) - rollout_corr/log_ppl_diff_max:np.float64(0.00896558165550232) - rollout_corr/log_ppl_diff_min:np.float64(-0.008629709482192993) - rollout_corr/ppl_ratio:np.float64(1.000638237921521) - rollout_corr/chi2_token:np.float64(0.0028806456830352545) - rollout_corr/chi2_seq:np.float64(0.6779269557446241) - actor/pg_loss:np.float64(0.004367844783701003) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004992729859623068) - actor/ppo_kl:np.float64(5.57020750591164e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(156417.83956303308) - self_distillation/token_reweight_w_std:np.float64(2357416.5479677515) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000363286351785) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0830252695013769) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.500577874481678) - perf/mfu/actor:np.float64(0.032587394156940674) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.82418060302734) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0022402419708669186 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0022402419708669186 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:284.21875 - response_length/max:788.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.21875 - response_length_non_aborted/max:788.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.90625 - prompt_length/max:631.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017307698726654053 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2802286799997091) - timing_s/agent_loop/generate_sequences/max:np.float64(5.627637119963765) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.104791782410757) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.627637119963765) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:423 - timing_s/agent_loop/slowest/response_length:788 - timing_s/gen:11.781217468902469 - timing_s/reward:0.05423154681921005 - timing_s/old_log_prob:2.465151973068714 - timing_s/adv:0.5479183085262775 - timing_s/update_actor:15.487478408962488 - timing_s/step:30.435431083664298 - timing_s/stop_profile:0.00011926889419555664 - timing_per_token_ms/adv:0.0039047769991895492 - timing_per_token_ms/gen:0.16191887670289265 - timing_per_token_ms/update_actor:0.11037256562829595 - perf/total_num_tokens:140320 - perf/time_per_step:30.435431083664298 - perf/throughput:576.3020064274463 (TaskRunner pid=2026458) Training Progress: 41%|████ | 41/100 [31:14<54:26, 55.37s/it] (TaskRunner pid=2026458) step:42 - global_seqlen/min:15278 - global_seqlen/max:20664 - global_seqlen/minmax_diff:5386 - global_seqlen/balanced_min:18018 - global_seqlen/balanced_max:18114 - global_seqlen/mean:18031.0 - actor/entropy:0.2842845022678375 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.289776086807251 - training/rollout_probs_diff_mean:0.005445326212793589 - training/rollout_probs_diff_std:0.013068844564259052 - training/rollout_actor_probs_pearson_corr:0.9985550045967102 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9997981786727905 - rollout_corr/rollout_is_ratio_fraction_high:1.2730420166917611e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.5460840333835222e-05 - rollout_corr/rollout_is_max:2.036485433578491 - rollout_corr/rollout_is_min:0.43032172322273254 - rollout_corr/rollout_is_std:0.04114258661866188 - rollout_corr/rollout_is_eff_sample_size:0.998309494689457 - rollout_corr/rollout_is_seq_mean:0.999789297580719 - rollout_corr/rollout_is_seq_std:0.0025704929139465094 - rollout_corr/rollout_is_seq_max:1.008382797241211 - rollout_corr/rollout_is_seq_min:0.9924810528755188 - rollout_corr/rollout_is_seq_max_deviation:0.008382797241210938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3495313432067633) - rollout_corr/training_log_ppl:np.float64(0.2967612352222204) - rollout_corr/kl:np.float64(0.0012985346789022856) - rollout_corr/k3_kl:np.float64(0.0010939320847001) - rollout_corr/rollout_ppl:np.float64(1.3477259748615324) - rollout_corr/rollout_log_ppl:np.float64(0.29546269873389974) - rollout_corr/log_ppl_diff:np.float64(0.0012985364883206785) - rollout_corr/log_ppl_abs_diff:np.float64(0.002426193736027926) - rollout_corr/log_ppl_diff_max:np.float64(0.00859573483467102) - rollout_corr/log_ppl_diff_min:np.float64(-0.006498366594314575) - rollout_corr/ppl_ratio:np.float64(1.0013033242430538) - rollout_corr/chi2_token:np.float64(0.001749418443068862) - rollout_corr/chi2_seq:np.float64(0.668122704140842) - actor/pg_loss:np.float64(0.0077214669436216354) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001590086207670538) - actor/ppo_kl:np.float64(0.0002024844293835315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(176662.73121298593) - self_distillation/token_reweight_w_std:np.float64(2621435.3882032083) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9981021112762392) - self_distillation/token_reweight_w_clip_frac:np.float64(0.14360627194400877) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.6128042787313461) - perf/mfu/actor:np.float64(0.03360615388418303) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.27114486694336) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003445170121267438 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003445170121267438 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:306.84375 - response_length/max:965.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:306.84375 - response_length_non_aborted/max:965.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:256.625 - prompt_length/max:557.0 - prompt_length/min:175.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013544969260692596 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3969923835247755) - timing_s/agent_loop/generate_sequences/max:np.float64(6.683027235791087) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.339469947764883) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.683027235791087) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:965 - timing_s/gen:13.290308959782124 - timing_s/reward:0.05642969720065594 - timing_s/old_log_prob:2.2229634914547205 - timing_s/adv:0.47765903919935226 - timing_s/update_actor:15.349972175434232 - timing_s/step:31.484555166214705 - timing_s/stop_profile:3.08305025100708e-05 - timing_per_token_ms/adv:0.003311373739666077 - timing_per_token_ms/gen:0.16919122313603885 - timing_per_token_ms/update_actor:0.10641376085238084 - perf/total_num_tokens:144248 - perf/time_per_step:31.484555166214705 - perf/throughput:572.6934970117862 (TaskRunner pid=2026458) Training Progress: 42%|████▏ | 42/100 [31:46<46:36, 48.21s/it] (TaskRunner pid=2026458) step:43 - global_seqlen/min:17746 - global_seqlen/max:21005 - global_seqlen/minmax_diff:3259 - global_seqlen/balanced_min:19023 - global_seqlen/balanced_max:19027 - global_seqlen/mean:19025.375 - actor/entropy:0.29123467206954956 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.277299165725708 - training/rollout_probs_diff_mean:0.0057848598808050156 - training/rollout_probs_diff_std:0.013805463910102844 - training/rollout_actor_probs_pearson_corr:0.9984029531478882 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.62890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9998417496681213 - rollout_corr/rollout_is_ratio_fraction_high:1.3395668247540016e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.697834032820538e-05 - rollout_corr/rollout_is_max:2.0824544429779053 - rollout_corr/rollout_is_min:0.3074188232421875 - rollout_corr/rollout_is_std:0.0433402918279171 - rollout_corr/rollout_is_eff_sample_size:0.9981245471106103 - rollout_corr/rollout_is_seq_mean:0.9999098181724548 - rollout_corr/rollout_is_seq_std:0.0025561333168298006 - rollout_corr/rollout_is_seq_max:1.0072154998779297 - rollout_corr/rollout_is_seq_min:0.9910159111022949 - rollout_corr/rollout_is_seq_max_deviation:0.008984088897705078 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3661411092616618) - rollout_corr/training_log_ppl:np.float64(0.30844287818763405) - rollout_corr/kl:np.float64(0.001555773359804391) - rollout_corr/k3_kl:np.float64(0.0014595502659631165) - rollout_corr/rollout_ppl:np.float64(1.3639800301752985) - rollout_corr/rollout_log_ppl:np.float64(0.30688710472895764) - rollout_corr/log_ppl_diff:np.float64(0.0015557734586764127) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026391229184810072) - rollout_corr/log_ppl_diff_max:np.float64(0.011147260665893555) - rollout_corr/log_ppl_diff_min:np.float64(-0.006583303213119507) - rollout_corr/ppl_ratio:np.float64(1.0015615709125996) - rollout_corr/chi2_token:np.float64(0.0026983728166669607) - rollout_corr/chi2_seq:np.float64(0.5271497289650142) - actor/pg_loss:np.float64(0.0038334736600518227) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013588511892521637) - actor/ppo_kl:np.float64(0.00048035153239256445) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62890625) - self_distillation/token_reweight_w_mean:np.float64(120698.68936810177) - self_distillation/token_reweight_w_std:np.float64(1773706.5877801427) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000403418438509) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07702912960667163) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.46645358949899673) - perf/mfu/actor:np.float64(0.03541705134193752) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.77776336669922) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:0 - critic/score/mean:0.62890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00016074800805654377 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00016074800805654377 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.60546875 - response_length/max:660.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.60546875 - response_length_non_aborted/max:660.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:302.9375 - prompt_length/max:813.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.972890019416809e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.429830538108945) - timing_s/agent_loop/generate_sequences/max:np.float64(5.387127343565226) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1587736944129574) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.387127343565226) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:660 - timing_s/gen:11.481718624010682 - timing_s/reward:0.056908562779426575 - timing_s/old_log_prob:2.2525762263685465 - timing_s/adv:0.5500077605247498 - timing_s/update_actor:15.172130836173892 - timing_s/step:29.60062885284424 - timing_s/stop_profile:0.00011467933654785156 - timing_per_token_ms/adv:0.0036136459894006673 - timing_per_token_ms/gen:0.15380528893130277 - timing_per_token_ms/update_actor:0.09968352027341046 - perf/total_num_tokens:152203 - perf/time_per_step:29.60062885284424 - perf/throughput:642.7355004713661 (TaskRunner pid=2026458) Training Progress: 43%|████▎ | 43/100 [32:15<40:30, 42.64s/it] (TaskRunner pid=2026458) step:44 - global_seqlen/min:16682 - global_seqlen/max:21465 - global_seqlen/minmax_diff:4783 - global_seqlen/balanced_min:19287 - global_seqlen/balanced_max:19289 - global_seqlen/mean:19287.75 - actor/entropy:0.2704760432243347 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26803696155548096 - training/rollout_probs_diff_mean:0.005588939879089594 - training/rollout_probs_diff_std:0.013731644488871098 - training/rollout_actor_probs_pearson_corr:0.9983340501785278 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000927448272705 - rollout_corr/rollout_is_ratio_fraction_high:2.68305120698642e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.049153257161379e-05 - rollout_corr/rollout_is_max:2.0968687534332275 - rollout_corr/rollout_is_min:0.22089464962482452 - rollout_corr/rollout_is_std:0.04294377565383911 - rollout_corr/rollout_is_eff_sample_size:0.9981595832973885 - rollout_corr/rollout_is_seq_mean:1.0000972747802734 - rollout_corr/rollout_is_seq_std:0.00284771085716784 - rollout_corr/rollout_is_seq_max:1.0088731050491333 - rollout_corr/rollout_is_seq_min:0.9906178116798401 - rollout_corr/rollout_is_seq_max_deviation:0.009382188320159912 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3369123949669302) - rollout_corr/training_log_ppl:np.float64(0.2868449309025891) - rollout_corr/kl:np.float64(0.0009355866875218055) - rollout_corr/k3_kl:np.float64(0.0010745994904937106) - rollout_corr/rollout_ppl:np.float64(1.335645436309278) - rollout_corr/rollout_log_ppl:np.float64(0.28590934313251637) - rollout_corr/log_ppl_diff:np.float64(0.0009355877700727433) - rollout_corr/log_ppl_abs_diff:np.float64(0.002383651997661218) - rollout_corr/log_ppl_diff_max:np.float64(0.011576682329177856) - rollout_corr/log_ppl_diff_min:np.float64(-0.006330162286758423) - rollout_corr/ppl_ratio:np.float64(1.00094054476358) - rollout_corr/chi2_token:np.float64(0.00241070962511003) - rollout_corr/chi2_seq:np.float64(0.8048715838231146) - actor/pg_loss:np.float64(0.0025291290367022157) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006139230463304557) - actor/ppo_kl:np.float64(5.7461318307527876e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(186553.62624621042) - self_distillation/token_reweight_w_std:np.float64(2401896.473150722) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0013129995204508) - self_distillation/token_reweight_w_clip_frac:np.float64(0.05587577508413233) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.40642886608839035) - perf/mfu/actor:np.float64(0.03559055533852201) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.5995101928711) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:0 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0029178180266171694 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0029178180266171694 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.1796875 - response_length/max:674.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.1796875 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:311.5625 - prompt_length/max:701.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001512337476015091 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5085251331329346) - timing_s/agent_loop/generate_sequences/max:np.float64(5.416358392685652) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1938710649119457) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.416358392685652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:634 - timing_s/gen:11.832684468477964 - timing_s/reward:0.057033905759453773 - timing_s/old_log_prob:2.2766003496944904 - timing_s/adv:0.5443154312670231 - timing_s/update_actor:15.36143240518868 - timing_s/step:30.159771636128426 - timing_s/stop_profile:0.0001326482743024826 - timing_per_token_ms/adv:0.003527598030271954 - timing_per_token_ms/gen:0.1587384892876226 - timing_per_token_ms/update_actor:0.0995543311505274 - perf/total_num_tokens:154302 - perf/time_per_step:30.159771636128426 - perf/throughput:639.5190995708728 (TaskRunner pid=2026458) Training Progress: 44%|████▍ | 44/100 [32:46<36:19, 38.91s/it] (TaskRunner pid=2026458) step:45 - global_seqlen/min:15729 - global_seqlen/max:24872 - global_seqlen/minmax_diff:9143 - global_seqlen/balanced_min:18453 - global_seqlen/balanced_max:18459 - global_seqlen/mean:18456.25 - actor/entropy:0.28802311420440674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29810237884521484 - training/rollout_probs_diff_mean:0.005760597065091133 - training/rollout_probs_diff_std:0.013836598955094814 - training/rollout_actor_probs_pearson_corr:0.9983675479888916 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:1.000093698501587 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.471605279832147e-05 - rollout_corr/rollout_is_max:1.7223904132843018 - rollout_corr/rollout_is_min:0.3712316155433655 - rollout_corr/rollout_is_std:0.04281449690461159 - rollout_corr/rollout_is_eff_sample_size:0.9981707478872216 - rollout_corr/rollout_is_seq_mean:1.000179648399353 - rollout_corr/rollout_is_seq_std:0.002977126045152545 - rollout_corr/rollout_is_seq_max:1.008954644203186 - rollout_corr/rollout_is_seq_min:0.9924071431159973 - rollout_corr/rollout_is_seq_max_deviation:0.008954644203186035 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3517238958738744) - rollout_corr/training_log_ppl:np.float64(0.2982309421640821) - rollout_corr/kl:np.float64(0.0008551144521504739) - rollout_corr/k3_kl:np.float64(0.0011088182236562716) - rollout_corr/rollout_ppl:np.float64(1.350550597999245) - rollout_corr/rollout_log_ppl:np.float64(0.29737582663074136) - rollout_corr/log_ppl_diff:np.float64(0.0008551155333407223) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025770837091840804) - rollout_corr/log_ppl_diff_max:np.float64(0.009607642889022827) - rollout_corr/log_ppl_diff_min:np.float64(-0.007143139839172363) - rollout_corr/ppl_ratio:np.float64(1.000860691536218) - rollout_corr/chi2_token:np.float64(0.002733499277383089) - rollout_corr/chi2_seq:np.float64(0.8130777420010418) - actor/pg_loss:np.float64(0.002860508277080953) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007193945029939641) - actor/ppo_kl:np.float64(8.01470097968604e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(108369.78586033895) - self_distillation/token_reweight_w_std:np.float64(1441947.6739233742) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9984894776716828) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07410955597879365) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.417113721370697) - perf/mfu/actor:np.float64(0.03440527561058198) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.48589324951172) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:0 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00041734983096830547 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00041734983096830547 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:262.0703125 - response_length/max:648.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:262.0703125 - response_length_non_aborted/max:648.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:314.6875 - prompt_length/max:845.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.201839566230774e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3249220605939627) - timing_s/agent_loop/generate_sequences/max:np.float64(5.273081086575985) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.846716286490846) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.273081086575985) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:418 - timing_s/agent_loop/slowest/response_length:648 - timing_s/gen:11.049067402258515 - timing_s/reward:0.0605863519012928 - timing_s/old_log_prob:2.241894433274865 - timing_s/adv:0.5028043333441019 - timing_s/update_actor:15.405420746654272 - timing_s/step:29.348894773051143 - timing_s/stop_profile:0.00011529400944709778 - timing_per_token_ms/adv:0.0034053798397839616 - timing_per_token_ms/gen:0.16469022808553457 - timing_per_token_ms/update_actor:0.1043374246302355 - perf/total_num_tokens:147650 - perf/time_per_step:29.348894773051143 - perf/throughput:628.8567301330532 (TaskRunner pid=2026458) Training Progress: 45%|████▌ | 45/100 [33:15<33:03, 36.06s/it] (TaskRunner pid=2026458) step:46 - global_seqlen/min:16485 - global_seqlen/max:20444 - global_seqlen/minmax_diff:3959 - global_seqlen/balanced_min:17819 - global_seqlen/balanced_max:17823 - global_seqlen/mean:17821.125 - actor/entropy:0.29167261719703674 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032017827033997 - training/rollout_probs_diff_mean:0.005424427799880505 - training/rollout_probs_diff_std:0.013135196641087532 - training/rollout_actor_probs_pearson_corr:0.9985377788543701 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.54296875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9998041391372681 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7731173759093508e-05 - rollout_corr/rollout_is_max:1.7920777797698975 - rollout_corr/rollout_is_min:0.19150957465171814 - rollout_corr/rollout_is_std:0.03981738165020943 - rollout_corr/rollout_is_eff_sample_size:0.9984166697314456 - rollout_corr/rollout_is_seq_mean:0.9997422695159912 - rollout_corr/rollout_is_seq_std:0.0028411736711859703 - rollout_corr/rollout_is_seq_max:1.0074349641799927 - rollout_corr/rollout_is_seq_min:0.9918615221977234 - rollout_corr/rollout_is_seq_max_deviation:0.008138477802276611 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3577020564116538) - rollout_corr/training_log_ppl:np.float64(0.3013367626699619) - rollout_corr/kl:np.float64(0.0011281410026526828) - rollout_corr/k3_kl:np.float64(0.000916888057986398) - rollout_corr/rollout_ppl:np.float64(1.3561205281876028) - rollout_corr/rollout_log_ppl:np.float64(0.3002086222113576) - rollout_corr/log_ppl_diff:np.float64(0.0011281404586043209) - rollout_corr/log_ppl_abs_diff:np.float64(0.00256365115637891) - rollout_corr/log_ppl_diff_max:np.float64(0.008949041366577148) - rollout_corr/log_ppl_diff_min:np.float64(-0.009114667773246765) - rollout_corr/ppl_ratio:np.float64(1.0011336989700794) - rollout_corr/chi2_token:np.float64(0.0014092018827795982) - rollout_corr/chi2_seq:np.float64(0.8297181797679514) - actor/pg_loss:np.float64(0.005181095330044627) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006262393026190693) - actor/ppo_kl:np.float64(4.9855951789634645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.54296875) - self_distillation/token_reweight_w_mean:np.float64(97563.27732459153) - self_distillation/token_reweight_w_std:np.float64(1470502.0369112268) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995444938540459) - self_distillation/token_reweight_w_clip_frac:np.float64(0.10749436303740367) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.4946809858083725) - perf/mfu/actor:np.float64(0.03312182529882031) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.53104400634766) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:0 - critic/score/mean:0.54296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.54296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002026108792051673 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002026108792051673 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:281.72265625 - response_length/max:593.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.72265625 - response_length_non_aborted/max:593.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.1875 - prompt_length/max:595.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.71662050485611e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3297640141099691) - timing_s/agent_loop/generate_sequences/max:np.float64(5.09482192248106) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.099804258738004) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.09482192248106) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:593 - timing_s/gen:11.312629146501422 - timing_s/reward:0.05594059266149998 - timing_s/old_log_prob:2.290945826098323 - timing_s/adv:0.4917367957532406 - timing_s/update_actor:15.34849851578474 - timing_s/step:29.586808905005455 - timing_s/stop_profile:0.00011978484690189362 - timing_per_token_ms/adv:0.0034491144340862362 - timing_per_token_ms/gen:0.15685624362531608 - timing_per_token_ms/update_actor:0.10765663303933352 - perf/total_num_tokens:142569 - perf/time_per_step:29.586808905005455 - perf/throughput:602.3334607398315 (TaskRunner pid=2026458) Training Progress: 46%|████▌ | 46/100 [33:45<30:43, 34.13s/it] (TaskRunner pid=2026458) step:47 - global_seqlen/min:14094 - global_seqlen/max:21506 - global_seqlen/minmax_diff:7412 - global_seqlen/balanced_min:18641 - global_seqlen/balanced_max:18644 - global_seqlen/mean:18642.625 - actor/entropy:0.27096572518348694 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7320154905319214 - training/rollout_probs_diff_mean:0.005300942808389664 - training/rollout_probs_diff_std:0.015477053821086884 - training/rollout_actor_probs_pearson_corr:0.9979209303855896 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0002624988555908 - rollout_corr/rollout_is_ratio_fraction_high:1.350238289887784e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00017553097859490663 - rollout_corr/rollout_is_max:5.250345230102539 - rollout_corr/rollout_is_min:0.1398765593767166 - rollout_corr/rollout_is_std:0.041101276874542236 - rollout_corr/rollout_is_eff_sample_size:0.9983143657831011 - rollout_corr/rollout_is_seq_mean:1.0002628564834595 - rollout_corr/rollout_is_seq_std:0.0028906052466481924 - rollout_corr/rollout_is_seq_max:1.0110986232757568 - rollout_corr/rollout_is_seq_min:0.9913266897201538 - rollout_corr/rollout_is_seq_max_deviation:0.011098623275756836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.33038782607764) - rollout_corr/training_log_ppl:np.float64(0.2808810631977394) - rollout_corr/kl:np.float64(0.0008357813613741882) - rollout_corr/k3_kl:np.float64(0.0010391420327664491) - rollout_corr/rollout_ppl:np.float64(1.329236305784434) - rollout_corr/rollout_log_ppl:np.float64(0.28004528186284006) - rollout_corr/log_ppl_diff:np.float64(0.0008357813348993659) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023171580396592617) - rollout_corr/log_ppl_diff_max:np.float64(0.011047214269638062) - rollout_corr/log_ppl_diff_min:np.float64(-0.006683439016342163) - rollout_corr/ppl_ratio:np.float64(1.0008402362000197) - rollout_corr/chi2_token:np.float64(0.002344296546652913) - rollout_corr/chi2_seq:np.float64(0.7856939071789384) - actor/pg_loss:np.float64(0.0035682206507772207) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00080771961438586) - actor/ppo_kl:np.float64(0.000137977912149978) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(180196.61803952185) - self_distillation/token_reweight_w_std:np.float64(1850028.0704001258) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9998111962340772) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09769563036388718) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.4677669107913971) - perf/mfu/actor:np.float64(0.03410809941893227) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.31404495239258) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003762101521715522 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003762101521715522 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:289.30078125 - response_length/max:738.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:289.30078125 - response_length_non_aborted/max:738.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:293.28125 - prompt_length/max:950.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.999237954616547e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5701810363680124) - timing_s/agent_loop/generate_sequences/max:np.float64(5.741097932681441) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.086432929841976) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.741097932681441) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:738 - timing_s/gen:12.44257121346891 - timing_s/reward:0.11020655557513237 - timing_s/old_log_prob:3.6139515712857246 - timing_s/adv:0.5418413802981377 - timing_s/update_actor:15.522931521758437 - timing_s/step:32.423666927963495 - timing_s/stop_profile:0.00011558458209037781 - timing_per_token_ms/adv:0.0036330813143142238 - timing_per_token_ms/gen:0.16800436415210313 - timing_per_token_ms/update_actor:0.10408225452262247 - perf/total_num_tokens:149141 - perf/time_per_step:32.423666927963495 - perf/throughput:574.9696677250851 (TaskRunner pid=2026458) Training Progress: 47%|████▋ | 47/100 [34:17<29:42, 33.63s/it] (TaskRunner pid=2026458) step:48 - global_seqlen/min:14172 - global_seqlen/max:18922 - global_seqlen/minmax_diff:4750 - global_seqlen/balanced_min:17273 - global_seqlen/balanced_max:17276 - global_seqlen/mean:17275.125 - actor/entropy:0.28335925936698914 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2923179566860199 - training/rollout_probs_diff_mean:0.005517815239727497 - training/rollout_probs_diff_std:0.0134055744856596 - training/rollout_actor_probs_pearson_corr:0.9984236359596252 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999392032623291 - rollout_corr/rollout_is_ratio_fraction_high:1.4249682863010094e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.8499365726020187e-05 - rollout_corr/rollout_is_max:2.0768961906433105 - rollout_corr/rollout_is_min:0.30299994349479675 - rollout_corr/rollout_is_std:0.04144066944718361 - rollout_corr/rollout_is_eff_sample_size:0.9982854963367315 - rollout_corr/rollout_is_seq_mean:0.9998867511749268 - rollout_corr/rollout_is_seq_std:0.002759289462119341 - rollout_corr/rollout_is_seq_max:1.0119271278381348 - rollout_corr/rollout_is_seq_min:0.9927634000778198 - rollout_corr/rollout_is_seq_max_deviation:0.011927127838134766 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3371913330629468) - rollout_corr/training_log_ppl:np.float64(0.28653379750903696) - rollout_corr/kl:np.float64(0.0009971100477166317) - rollout_corr/k3_kl:np.float64(0.0009379864269476457) - rollout_corr/rollout_ppl:np.float64(1.3358645238913596) - rollout_corr/rollout_log_ppl:np.float64(0.28553668738459237) - rollout_corr/log_ppl_diff:np.float64(0.000997110124444589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023510434839408845) - rollout_corr/log_ppl_diff_max:np.float64(0.009961530566215515) - rollout_corr/log_ppl_diff_min:np.float64(-0.00725024938583374) - rollout_corr/ppl_ratio:np.float64(1.0010016579180956) - rollout_corr/chi2_token:np.float64(0.0017369601409882307) - rollout_corr/chi2_seq:np.float64(0.5906375367194414) - actor/pg_loss:np.float64(0.0036190334940329194) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005808295049973822) - actor/ppo_kl:np.float64(-3.3437734998642554e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(104527.51239224058) - self_distillation/token_reweight_w_std:np.float64(1541202.450584725) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000157124362886) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09064709232188761) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.42223525047302246) - perf/mfu/actor:np.float64(0.031950463769025786) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.0163230895996) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:0 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019717998802661896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019717998802661896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:274.12890625 - response_length/max:573.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:274.12890625 - response_length_non_aborted/max:573.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.71875 - prompt_length/max:563.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012047216296195984 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2815109174698591) - timing_s/agent_loop/generate_sequences/max:np.float64(4.910025550052524) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0799834246499813) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.910025550052524) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:200 - timing_s/agent_loop/slowest/response_length:573 - timing_s/gen:10.97082038037479 - timing_s/reward:0.05485686659812927 - timing_s/old_log_prob:2.286929327994585 - timing_s/adv:0.5000464376062155 - timing_s/update_actor:15.382348308339715 - timing_s/step:29.284351360052824 - timing_s/stop_profile:0.0001140497624874115 - timing_per_token_ms/adv:0.0036182548433529096 - timing_per_token_ms/gen:0.15633071206199736 - timing_per_token_ms/update_actor:0.11130417513867277 - perf/total_num_tokens:138201 - perf/time_per_step:29.284351360052824 - perf/throughput:589.9097708397677 (TaskRunner pid=2026458) Training Progress: 48%|████▊ | 48/100 [34:46<28:01, 32.34s/it] (TaskRunner pid=2026458) step:49 - global_seqlen/min:17224 - global_seqlen/max:19716 - global_seqlen/minmax_diff:2492 - global_seqlen/balanced_min:18546 - global_seqlen/balanced_max:18550 - global_seqlen/mean:18548.75 - actor/entropy:0.27977773547172546 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3583575189113617 - training/rollout_probs_diff_mean:0.005447280127555132 - training/rollout_probs_diff_std:0.013272756710648537 - training/rollout_actor_probs_pearson_corr:0.9984596967697144 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.77734375 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.77734375 - rollout_corr/rollout_is_mean:0.9998903274536133 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.59693042607978e-05 - rollout_corr/rollout_is_max:1.740144968032837 - rollout_corr/rollout_is_min:0.2820626199245453 - rollout_corr/rollout_is_std:0.04080290347337723 - rollout_corr/rollout_is_eff_sample_size:0.9983375338303951 - rollout_corr/rollout_is_seq_mean:0.9998434782028198 - rollout_corr/rollout_is_seq_std:0.0024030529893934727 - rollout_corr/rollout_is_seq_max:1.0073314905166626 - rollout_corr/rollout_is_seq_min:0.9888448715209961 - rollout_corr/rollout_is_seq_max_deviation:0.011155128479003906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3396195233799517) - rollout_corr/training_log_ppl:np.float64(0.28772373546962626) - rollout_corr/kl:np.float64(0.0007274425354566461) - rollout_corr/k3_kl:np.float64(0.0009657581270516857) - rollout_corr/rollout_ppl:np.float64(1.338642387650907) - rollout_corr/rollout_log_ppl:np.float64(0.2869962909608148) - rollout_corr/log_ppl_diff:np.float64(0.0007274445088114589) - rollout_corr/log_ppl_abs_diff:np.float64(0.002191261184634641) - rollout_corr/log_ppl_diff_max:np.float64(0.011016637086868286) - rollout_corr/log_ppl_diff_min:np.float64(-0.009674102067947388) - rollout_corr/ppl_ratio:np.float64(1.0007315212860703) - rollout_corr/chi2_token:np.float64(0.002396710915490985) - rollout_corr/chi2_seq:np.float64(1.0418477742932737) - actor/pg_loss:np.float64(0.003264832077547908) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005003044952900382) - actor/ppo_kl:np.float64(-0.0002956932246789279) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.77734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.77734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(133482.32340499433) - self_distillation/token_reweight_w_std:np.float64(1837865.312099801) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9993252530694008) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07909147051395848) - self_distillation/empty_target_batch:np.float64(0.22265625) - actor/grad_norm:np.float64(0.44828861951828003) - perf/mfu/actor:np.float64(0.03315053423820138) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(151.27535438537598) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005617484916001558 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005617484916001558 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:300.8359375 - response_length/max:722.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:300.8359375 - response_length_non_aborted/max:722.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.8125 - prompt_length/max:418.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001618172973394394 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2382317576557398) - timing_s/agent_loop/generate_sequences/max:np.float64(5.747685985639691) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2742665582336485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.747685985639691) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:358 - timing_s/agent_loop/slowest/response_length:722 - timing_s/gen:11.949256183579564 - timing_s/reward:0.0566436555236578 - timing_s/old_log_prob:2.331390691921115 - timing_s/adv:0.5100956819951534 - timing_s/update_actor:16.06329415179789 - timing_s/step:30.9992211535573 - timing_s/stop_profile:0.00011705979704856873 - timing_per_token_ms/adv:0.003437534079083182 - timing_per_token_ms/gen:0.15515693488949495 - timing_per_token_ms/update_actor:0.1082505165563575 - perf/total_num_tokens:148390 - perf/time_per_step:30.9992211535573 - perf/throughput:598.3618074827485 (TaskRunner pid=2026458) Training Progress: 49%|████▉ | 49/100 [35:17<27:09, 31.95s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to identify which of the given options matches the structure of the reactant that would form this product through a chemical reaction. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a structure with a substituted benzene ring (Cc2ccc(Cl)cc2) and a trifluoromethyl group (C(F)(F)F). The correct reactant should include these key functional groups and match the overall structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a structure with a substituted benzene ring (c2ccccc2) and a nitro group (N), which is consistent with the product's structure. It also includes a chlorine atom and a trifluoromethyl group, making it a strong candidate for the correct reactant. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B includes a chlorine atom and a substituted benzene ring, but it lacks the trifluoromethyl group and does not match the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C includes a substituted benzene ring and a trifluoromethyl group, but it has a different functional group (an oxygen atom) that is not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D includes a substituted benzene ring and a trifluoromethyl group, but it has a different functional group (a double bond) that is not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it contains the necessary functional groups and matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2026458) step:50 - global_seqlen/min:15714 - global_seqlen/max:24076 - global_seqlen/minmax_diff:8362 - global_seqlen/balanced_min:18055 - global_seqlen/balanced_max:18058 - global_seqlen/mean:18056.5 - actor/entropy:0.24377241730690002 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.36303967237472534 - training/rollout_probs_diff_mean:0.005148114170879126 - training/rollout_probs_diff_std:0.013693224638700485 - training/rollout_actor_probs_pearson_corr:0.9982551336288452 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.859375 - self_distillation/correct_sample_fraction:0.61328125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.859375 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:2.7458949261927046e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.864737224532291e-05 - rollout_corr/rollout_is_max:2.6678342819213867 - rollout_corr/rollout_is_min:0.17930372059345245 - rollout_corr/rollout_is_std:0.04035785421729088 - rollout_corr/rollout_is_eff_sample_size:0.9983741296547397 - rollout_corr/rollout_is_seq_mean:1.0000826120376587 - rollout_corr/rollout_is_seq_std:0.002955564297735691 - rollout_corr/rollout_is_seq_max:1.0125819444656372 - rollout_corr/rollout_is_seq_min:0.9909312725067139 - rollout_corr/rollout_is_seq_max_deviation:0.012581944465637207 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3021241263486445) - rollout_corr/training_log_ppl:np.float64(0.26025468575244304) - rollout_corr/kl:np.float64(0.0007726539755310569) - rollout_corr/k3_kl:np.float64(0.0010959870580791176) - rollout_corr/rollout_ppl:np.float64(1.3010970023460686) - rollout_corr/rollout_log_ppl:np.float64(0.2594820303056622) - rollout_corr/log_ppl_diff:np.float64(0.0007726554467808455) - rollout_corr/log_ppl_abs_diff:np.float64(0.00248320066020824) - rollout_corr/log_ppl_diff_max:np.float64(0.010783776640892029) - rollout_corr/log_ppl_diff_min:np.float64(-0.014668971300125122) - rollout_corr/ppl_ratio:np.float64(1.0007780094165355) - rollout_corr/chi2_token:np.float64(0.002904584165662527) - rollout_corr/chi2_seq:np.float64(1.7037486750632524) - actor/pg_loss:np.float64(0.00390949088614434) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012861561140198319) - actor/ppo_kl:np.float64(-2.874969411337247e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.61328125) - self_distillation/token_reweight_w_mean:np.float64(303182.6307455059) - self_distillation/token_reweight_w_std:np.float64(3853282.430746559) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001452625496313) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08799264291883446) - self_distillation/empty_target_batch:np.float64(0.140625) - actor/grad_norm:np.float64(0.9146713092923164) - perf/mfu/actor:np.float64(0.03308314844340515) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.47300720214844) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/reward/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08550843930681744) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8024761904761903) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05941417072300563) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6757285714285716) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06458122232163772) - val-aux/sciknoweval/score/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/score/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/score/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08550843930681744) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8024761904761903) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05941417072300563) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6757285714285716) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06458122232163772) - val-core/sciknoweval/acc/mean@16:np.float64(0.6586309523809524) - val-aux/sciknoweval/acc/std@16:np.float64(0.1466147179951147) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7180000000000002) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10968372523717904) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5997904761904762) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1360982101557332) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6585333333333333) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.146035780800308) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7595095238095239) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.07337309476992995) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5448142857142857) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11407729981266174) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.669457142857143) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.11545652344653062) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7867285714285714) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04348485587197246) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4990380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08739882833894748) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.674152380952381) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08550843930681744) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8024761904761903) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02280799651519509) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4621714285714286) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05941417072300563) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6757285714285716) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06458122232163772) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:0 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00038099291850812733 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00038099291850812733 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:284.515625 - response_length/max:754.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.515625 - response_length_non_aborted/max:754.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.75 - prompt_length/max:900.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001567751169204712 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4153082128614187) - timing_s/agent_loop/generate_sequences/max:np.float64(6.037689724937081) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1098132879196783) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.037689724937081) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:216 - timing_s/agent_loop/slowest/response_length:754 - timing_s/gen:12.157338604331017 - timing_s/reward:0.07574324123561382 - timing_s/old_log_prob:2.3888504561036825 - timing_s/adv:0.52120690792799 - timing_s/update_actor:15.507932955399156 - timing_s/step:30.738427244126797 - timing_s/testing:64.3304407633841 - timing_s/save_checkpoint:6.778321286663413 - timing_s/stop_profile:0.00014533661305904388 - timing_per_token_ms/adv:0.0036081667815467417 - timing_per_token_ms/gen:0.16691386957453755 - timing_per_token_ms/update_actor:0.10735699717137288 - perf/total_num_tokens:144452 - perf/time_per_step:30.738427244126797 - perf/throughput:587.4243290521658 (TaskRunner pid=2026458) Training Progress: 50%|█████ | 50/100 [36:59<44:06, 52.94s/it] (TaskRunner pid=2026458) step:51 - global_seqlen/min:17642 - global_seqlen/max:21320 - global_seqlen/minmax_diff:3678 - global_seqlen/balanced_min:19266 - global_seqlen/balanced_max:19274 - global_seqlen/mean:19272.125 - actor/entropy:0.27418410778045654 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3186328411102295 - training/rollout_probs_diff_mean:0.005429690238088369 - training/rollout_probs_diff_std:0.013169948011636734 - training/rollout_actor_probs_pearson_corr:0.9984568953514099 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8046875 - self_distillation/correct_sample_fraction:0.5390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8046875 - rollout_corr/rollout_is_mean:0.9999269247055054 - rollout_corr/rollout_is_ratio_fraction_high:2.6965442884829827e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6965442884829827e-05 - rollout_corr/rollout_is_max:2.9393632411956787 - rollout_corr/rollout_is_min:0.18107180297374725 - rollout_corr/rollout_is_std:0.04137301445007324 - rollout_corr/rollout_is_eff_sample_size:0.9982909612066878 - rollout_corr/rollout_is_seq_mean:0.9999714493751526 - rollout_corr/rollout_is_seq_std:0.0026941560208797455 - rollout_corr/rollout_is_seq_max:1.0119987726211548 - rollout_corr/rollout_is_seq_min:0.9906357526779175 - rollout_corr/rollout_is_seq_max_deviation:0.011998772621154785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3326939661055803) - rollout_corr/training_log_ppl:np.float64(0.283328754856484) - rollout_corr/kl:np.float64(0.001354399174752885) - rollout_corr/k3_kl:np.float64(0.0011438059663078093) - rollout_corr/rollout_ppl:np.float64(1.3308947938494384) - rollout_corr/rollout_log_ppl:np.float64(0.2819743549771374) - rollout_corr/log_ppl_diff:np.float64(0.0013543998793466017) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025673647614894435) - rollout_corr/log_ppl_diff_max:np.float64(0.017022624611854553) - rollout_corr/log_ppl_diff_min:np.float64(-0.007628902792930603) - rollout_corr/ppl_ratio:np.float64(1.001360428519547) - rollout_corr/chi2_token:np.float64(0.0018329326994717121) - rollout_corr/chi2_seq:np.float64(1.2063880120404065) - actor/pg_loss:np.float64(0.006517422734759748) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015835570143281075) - actor/ppo_kl:np.float64(0.0004230728660630234) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8046875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8046875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5390625) - self_distillation/token_reweight_w_mean:np.float64(238186.0888677421) - self_distillation/token_reweight_w_std:np.float64(3422829.743828913) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0022471062839031) - self_distillation/token_reweight_w_clip_frac:np.float64(0.1062716192973312) - self_distillation/empty_target_batch:np.float64(0.1953125) - actor/grad_norm:np.float64(0.5796995311975479) - perf/mfu/actor:np.float64(0.035632098241036) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.68563079833984) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:0 - critic/score/mean:0.5390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.010413717478513718 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.010413717478513718 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:289.72265625 - response_length/max:773.0 - response_length/min:143.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:289.72265625 - response_length_non_aborted/max:773.0 - response_length_non_aborted/min:143.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.53125 - prompt_length/max:608.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014895200729370117 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6900891289114952) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7452072985470295) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1825576704068226) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7452072985470295) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:321 - timing_s/agent_loop/slowest/response_length:773 - timing_s/gen:11.57777032442391 - timing_s/reward:0.05630043335258961 - timing_s/old_log_prob:2.3727036882191896 - timing_s/adv:0.5643414072692394 - timing_s/update_actor:15.28370295651257 - timing_s/step:29.95019925571978 - timing_s/stop_profile:0.00013649649918079376 - timing_per_token_ms/adv:0.003660347569801199 - timing_per_token_ms/gen:0.15609985741244872 - timing_per_token_ms/update_actor:0.09913088824216693 - perf/total_num_tokens:154177 - perf/time_per_step:29.95019925571978 - perf/throughput:643.472346726357 (TaskRunner pid=2026458) Training Progress: 51%|█████ | 51/100 [37:29<37:36, 46.06s/it] (TaskRunner pid=2026458) step:52 - global_seqlen/min:14188 - global_seqlen/max:21038 - global_seqlen/minmax_diff:6850 - global_seqlen/balanced_min:18334 - global_seqlen/balanced_max:18335 - global_seqlen/mean:18334.5 - actor/entropy:0.2573181688785553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730220556259155 - training/rollout_probs_diff_mean:0.005135765764862299 - training/rollout_probs_diff_std:0.012896839529275894 - training/rollout_actor_probs_pearson_corr:0.9984732270240784 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.59375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9999938011169434 - rollout_corr/rollout_is_ratio_fraction_high:2.519780355214607e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.519780355214607e-05 - rollout_corr/rollout_is_max:2.1215898990631104 - rollout_corr/rollout_is_min:0.3974515199661255 - rollout_corr/rollout_is_std:0.040721017867326736 - rollout_corr/rollout_is_eff_sample_size:0.9983446626766718 - rollout_corr/rollout_is_seq_mean:0.9999977946281433 - rollout_corr/rollout_is_seq_std:0.0026824194937944412 - rollout_corr/rollout_is_seq_max:1.0067678689956665 - rollout_corr/rollout_is_seq_min:0.9906572103500366 - rollout_corr/rollout_is_seq_max_deviation:0.009342789649963379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3137918766587973) - rollout_corr/training_log_ppl:np.float64(0.2694710589275928) - rollout_corr/kl:np.float64(0.0010667287329408914) - rollout_corr/k3_kl:np.float64(0.0010078331736167456) - rollout_corr/rollout_ppl:np.float64(1.3123429850675166) - rollout_corr/rollout_log_ppl:np.float64(0.26840433149482124) - rollout_corr/log_ppl_diff:np.float64(0.0010667274327715859) - rollout_corr/log_ppl_abs_diff:np.float64(0.002380369245656766) - rollout_corr/log_ppl_diff_max:np.float64(0.010203629732131958) - rollout_corr/log_ppl_diff_min:np.float64(-0.006238669157028198) - rollout_corr/ppl_ratio:np.float64(1.0010714137461036) - rollout_corr/chi2_token:np.float64(0.001948290504515171) - rollout_corr/chi2_seq:np.float64(0.5412429641000926) - actor/pg_loss:np.float64(0.003885703510604799) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005619293583549734) - actor/ppo_kl:np.float64(0.00019392038652732424) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59375) - self_distillation/token_reweight_w_mean:np.float64(186537.93477050052) - self_distillation/token_reweight_w_std:np.float64(2011255.6988170468) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005899087991565) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06515720163588412) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4466311186552048) - perf/mfu/actor:np.float64(0.03402000847749656) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.35942840576172) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.01099569071084261 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.01099569071084261 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:310.046875 - response_length/max:700.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:310.046875 - response_length_non_aborted/max:700.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.90625 - prompt_length/max:453.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014230050146579742 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4891746453940868) - timing_s/agent_loop/generate_sequences/max:np.float64(5.560187539085746) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.207101171356044) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.560187539085746) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:700 - timing_s/gen:11.716578045859933 - timing_s/reward:0.05753312073647976 - timing_s/old_log_prob:2.249606389552355 - timing_s/adv:0.5425826162099838 - timing_s/update_actor:15.384762125089765 - timing_s/step:30.04255287349224 - timing_s/stop_profile:0.00013405084609985352 - timing_per_token_ms/adv:0.003699191525607351 - timing_per_token_ms/gen:0.14761601126165314 - timing_per_token_ms/update_actor:0.10488943061639099 - perf/total_num_tokens:146676 - perf/time_per_step:30.04255287349224 - perf/throughput:610.2843549017191 (TaskRunner pid=2026458) Training Progress: 52%|█████▏ | 52/100 [37:59<33:00, 41.27s/it] (TaskRunner pid=2026458) step:53 - global_seqlen/min:17379 - global_seqlen/max:20313 - global_seqlen/minmax_diff:2934 - global_seqlen/balanced_min:18481 - global_seqlen/balanced_max:18482 - global_seqlen/mean:18481.75 - actor/entropy:0.278646856546402 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.18462908267974854 - training/rollout_probs_diff_mean:0.005563129670917988 - training/rollout_probs_diff_std:0.013384878635406494 - training/rollout_actor_probs_pearson_corr:0.9984221458435059 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:0.9999546408653259 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.5409334891010076e-05 - rollout_corr/rollout_is_max:1.8313579559326172 - rollout_corr/rollout_is_min:0.0737028419971466 - rollout_corr/rollout_is_std:0.04174019396305084 - rollout_corr/rollout_is_eff_sample_size:0.9982605487774039 - rollout_corr/rollout_is_seq_mean:0.9999428391456604 - rollout_corr/rollout_is_seq_std:0.0028358809649944305 - rollout_corr/rollout_is_seq_max:1.0075714588165283 - rollout_corr/rollout_is_seq_min:0.9910898804664612 - rollout_corr/rollout_is_seq_max_deviation:0.008910119533538818 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.338077249005437) - rollout_corr/training_log_ppl:np.float64(0.2873227519157808) - rollout_corr/kl:np.float64(0.0011784572657376202) - rollout_corr/k3_kl:np.float64(0.0013956419594478575) - rollout_corr/rollout_ppl:np.float64(1.3364087981171906) - rollout_corr/rollout_log_ppl:np.float64(0.2861442923604045) - rollout_corr/log_ppl_diff:np.float64(0.001178459555376321) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027842955314554274) - rollout_corr/log_ppl_diff_max:np.float64(0.0132656991481781) - rollout_corr/log_ppl_diff_min:np.float64(-0.010441526770591736) - rollout_corr/ppl_ratio:np.float64(1.0011851615272462) - rollout_corr/chi2_token:np.float64(0.003180387197062373) - rollout_corr/chi2_seq:np.float64(1.2029992926400155) - actor/pg_loss:np.float64(0.003771678893826902) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009931258632605022) - actor/ppo_kl:np.float64(0.00019185108615538127) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(52868.786943790736) - self_distillation/token_reweight_w_std:np.float64(637934.6799204362) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990235811565071) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07278861297527328) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.44415684044361115) - perf/mfu/actor:np.float64(0.0346355208538403) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.01051712036133) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:0 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006060396321117878 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006060396321117878 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:281.9921875 - response_length/max:616.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:281.9921875 - response_length_non_aborted/max:616.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.5625 - prompt_length/max:532.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001376233994960785 - timing_s/agent_loop/generate_sequences/min:np.float64(1.366797188296914) - timing_s/agent_loop/generate_sequences/max:np.float64(5.209049364551902) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1154251298939926) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.209049364551902) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:227 - timing_s/agent_loop/slowest/response_length:616 - timing_s/gen:11.11843347735703 - timing_s/reward:0.05367938429117203 - timing_s/old_log_prob:2.252691153436899 - timing_s/adv:0.4951089806854725 - timing_s/update_actor:15.301375834271312 - timing_s/step:29.318861059844494 - timing_s/stop_profile:0.00011683069169521332 - timing_per_token_ms/adv:0.0033486343330953 - timing_per_token_ms/gen:0.15401625540042985 - timing_per_token_ms/update_actor:0.10348976581134979 - perf/total_num_tokens:147854 - perf/time_per_step:29.318861059844494 - perf/throughput:630.3706669326542 (TaskRunner pid=2026458) Training Progress: 53%|█████▎ | 53/100 [38:29<29:31, 37.70s/it] (TaskRunner pid=2026458) step:54 - global_seqlen/min:15111 - global_seqlen/max:18863 - global_seqlen/minmax_diff:3752 - global_seqlen/balanced_min:17356 - global_seqlen/balanced_max:17358 - global_seqlen/mean:17357.625 - actor/entropy:0.2270389199256897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2413564920425415 - training/rollout_probs_diff_mean:0.005126654170453548 - training/rollout_probs_diff_std:0.01343945786356926 - training/rollout_actor_probs_pearson_corr:0.9981903433799744 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.50390625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:1.0001782178878784 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.876001235563308e-05 - rollout_corr/rollout_is_max:1.8594493865966797 - rollout_corr/rollout_is_min:0.35453474521636963 - rollout_corr/rollout_is_std:0.04036819189786911 - rollout_corr/rollout_is_eff_sample_size:0.998373654367098 - rollout_corr/rollout_is_seq_mean:1.000128984451294 - rollout_corr/rollout_is_seq_std:0.002654197160154581 - rollout_corr/rollout_is_seq_max:1.0092252492904663 - rollout_corr/rollout_is_seq_min:0.9911713600158691 - rollout_corr/rollout_is_seq_max_deviation:0.009225249290466309 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2745554209686816) - rollout_corr/training_log_ppl:np.float64(0.23950671087368391) - rollout_corr/kl:np.float64(0.0011859710301269644) - rollout_corr/k3_kl:np.float64(0.0012516252674004136) - rollout_corr/rollout_ppl:np.float64(1.2729601715691388) - rollout_corr/rollout_log_ppl:np.float64(0.23832074020174332) - rollout_corr/log_ppl_diff:np.float64(0.001185970671940595) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027243185904808342) - rollout_corr/log_ppl_diff_max:np.float64(0.01128050684928894) - rollout_corr/log_ppl_diff_min:np.float64(-0.011040359735488892) - rollout_corr/ppl_ratio:np.float64(1.0011925832368433) - rollout_corr/chi2_token:np.float64(0.0025079476181417704) - rollout_corr/chi2_seq:np.float64(1.487264727940783) - actor/pg_loss:np.float64(0.0043368348851799965) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0017240041906916304) - actor/ppo_kl:np.float64(0.0004469881809043841) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.50390625) - self_distillation/token_reweight_w_mean:np.float64(275413.10884484975) - self_distillation/token_reweight_w_std:np.float64(3763164.204352789) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0020550324115902) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08829297052579932) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.5274116396903992) - perf/mfu/actor:np.float64(0.03254539451835227) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.84357070922852) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:0 - critic/score/mean:0.50390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.50390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0047957319766283035 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0047957319766283035 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:271.64453125 - response_length/max:650.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:271.64453125 - response_length_non_aborted/max:650.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.78125 - prompt_length/max:597.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.452683687210083e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3436731528490782) - timing_s/agent_loop/generate_sequences/max:np.float64(5.206967985257506) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.978192554124689) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.206967985257506) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:650 - timing_s/gen:10.970427636057138 - timing_s/reward:0.05443461053073406 - timing_s/old_log_prob:2.2816072031855583 - timing_s/adv:0.47508152201771736 - timing_s/update_actor:15.217902828007936 - timing_s/step:29.08791142515838 - timing_s/stop_profile:0.0001310836523771286 - timing_per_token_ms/adv:0.0034212739503367926 - timing_per_token_ms/gen:0.15775481566352423 - timing_per_token_ms/update_actor:0.10959090621562524 - perf/total_num_tokens:138861 - perf/time_per_step:29.08791142515838 - perf/throughput:596.7298492592096 (TaskRunner pid=2026458) Training Progress: 54%|█████▍ | 54/100 [38:58<26:55, 35.13s/it] (TaskRunner pid=2026458) step:55 - global_seqlen/min:15554 - global_seqlen/max:21932 - global_seqlen/minmax_diff:6378 - global_seqlen/balanced_min:18814 - global_seqlen/balanced_max:18817 - global_seqlen/mean:18816.625 - actor/entropy:0.22956356406211853 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9232618808746338 - training/rollout_probs_diff_mean:0.005393934901803732 - training/rollout_probs_diff_std:0.014924020506441593 - training/rollout_actor_probs_pearson_corr:0.9978342056274414 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.000108242034912 - rollout_corr/rollout_is_ratio_fraction_high:4.133313268539496e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.133313268539496e-05 - rollout_corr/rollout_is_max:2.1670196056365967 - rollout_corr/rollout_is_min:0.04885798692703247 - rollout_corr/rollout_is_std:0.042956262826919556 - rollout_corr/rollout_is_eff_sample_size:0.9981586331310626 - rollout_corr/rollout_is_seq_mean:1.0001986026763916 - rollout_corr/rollout_is_seq_std:0.002816676627844572 - rollout_corr/rollout_is_seq_max:1.0112348794937134 - rollout_corr/rollout_is_seq_min:0.9917329549789429 - rollout_corr/rollout_is_seq_max_deviation:0.011234879493713379 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2765053128823638) - rollout_corr/training_log_ppl:np.float64(0.24006979388650507) - rollout_corr/kl:np.float64(0.0010645344639215182) - rollout_corr/k3_kl:np.float64(0.0011318304883047858) - rollout_corr/rollout_ppl:np.float64(1.275088603142649) - rollout_corr/rollout_log_ppl:np.float64(0.23900525833596475) - rollout_corr/log_ppl_diff:np.float64(0.0010645355505403131) - rollout_corr/log_ppl_abs_diff:np.float64(0.002453350374707952) - rollout_corr/log_ppl_diff_max:np.float64(0.011057406663894653) - rollout_corr/log_ppl_diff_min:np.float64(-0.007540911436080933) - rollout_corr/ppl_ratio:np.float64(1.0010695066303015) - rollout_corr/chi2_token:np.float64(0.0023642820306122303) - rollout_corr/chi2_seq:np.float64(0.9924367545172572) - actor/pg_loss:np.float64(0.005413581035099924) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001406622541253455) - actor/ppo_kl:np.float64(0.0002698558086962066) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.609375) - self_distillation/token_reweight_w_mean:np.float64(295807.0609528662) - self_distillation/token_reweight_w_std:np.float64(3781227.560846627) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014416894409806) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09695331199327484) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5869030505418777) - perf/mfu/actor:np.float64(0.03432748799407685) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.77687454223633) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014518261887133121 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014518261887133121 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:283.51953125 - response_length/max:687.0 - response_length/min:138.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:283.51953125 - response_length_non_aborted/max:687.0 - response_length_non_aborted/min:138.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:304.5 - prompt_length/max:625.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010005757212638855 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9696572478860617) - timing_s/agent_loop/generate_sequences/max:np.float64(5.428103374317288) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.78926639315614) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.428103374317288) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:329 - timing_s/agent_loop/slowest/response_length:687 - timing_s/gen:11.274114403873682 - timing_s/reward:0.0568949319422245 - timing_s/old_log_prob:2.307572791352868 - timing_s/adv:0.5059547368437052 - timing_s/update_actor:15.750575561076403 - timing_s/step:29.98292907886207 - timing_s/stop_profile:0.00012236833572387695 - timing_per_token_ms/adv:0.003361088511115205 - timing_per_token_ms/gen:0.1553314834994514 - timing_per_token_ms/update_actor:0.10463204454223594 - perf/total_num_tokens:150533 - perf/time_per_step:29.98292907886207 - perf/throughput:627.5779444532556 (TaskRunner pid=2026458) Training Progress: 55%|█████▌ | 55/100 [39:28<25:11, 33.60s/it] (TaskRunner pid=2026458) step:56 - global_seqlen/min:15183 - global_seqlen/max:19968 - global_seqlen/minmax_diff:4785 - global_seqlen/balanced_min:16851 - global_seqlen/balanced_max:16856 - global_seqlen/mean:16853.75 - actor/entropy:0.23840856552124023 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5032567977905273 - training/rollout_probs_diff_mean:0.0051898774690926075 - training/rollout_probs_diff_std:0.013416245579719543 - training/rollout_actor_probs_pearson_corr:0.9982264637947083 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.0000948905944824 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.9873039238736965e-05 - rollout_corr/rollout_is_max:1.868481159210205 - rollout_corr/rollout_is_min:0.42863455414772034 - rollout_corr/rollout_is_std:0.03991232439875603 - rollout_corr/rollout_is_eff_sample_size:0.9984097775176272 - rollout_corr/rollout_is_seq_mean:1.0001156330108643 - rollout_corr/rollout_is_seq_std:0.002804977586492896 - rollout_corr/rollout_is_seq_max:1.008453607559204 - rollout_corr/rollout_is_seq_min:0.9917566776275635 - rollout_corr/rollout_is_seq_max_deviation:0.008453607559204102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.288645532913506) - rollout_corr/training_log_ppl:np.float64(0.25009057341958396) - rollout_corr/kl:np.float64(0.001021610375019577) - rollout_corr/k3_kl:np.float64(0.0012247983307815957) - rollout_corr/rollout_ppl:np.float64(1.2872521225363016) - rollout_corr/rollout_log_ppl:np.float64(0.2490689625556115) - rollout_corr/log_ppl_diff:np.float64(0.0010216108639724553) - rollout_corr/log_ppl_abs_diff:np.float64(0.002677871671039611) - rollout_corr/log_ppl_diff_max:np.float64(0.011799484491348267) - rollout_corr/log_ppl_diff_min:np.float64(-0.013348877429962158) - rollout_corr/ppl_ratio:np.float64(1.001028069993481) - rollout_corr/chi2_token:np.float64(0.0027951118536293507) - rollout_corr/chi2_seq:np.float64(0.911775927990675) - actor/pg_loss:np.float64(0.0038659967249259353) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001200501193125092) - actor/ppo_kl:np.float64(0.0002928259359418206) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(173579.36585247447) - self_distillation/token_reweight_w_std:np.float64(2190372.348648581) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999903631862253) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08299089543288574) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.5205514430999756) - perf/mfu/actor:np.float64(0.03136809590233023) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.65859985351562) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00015309933223761618 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00015309933223761618 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:261.5234375 - response_length/max:725.0 - response_length/min:131.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:261.5234375 - response_length_non_aborted/max:725.0 - response_length_non_aborted/min:131.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.15625 - prompt_length/max:458.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001024492084980011 - timing_s/agent_loop/generate_sequences/min:np.float64(1.564612640067935) - timing_s/agent_loop/generate_sequences/max:np.float64(5.375423099845648) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8240807638067054) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.375423099845648) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:298 - timing_s/agent_loop/slowest/response_length:725 - timing_s/gen:11.348324349150062 - timing_s/reward:0.054198144003748894 - timing_s/old_log_prob:2.395893217995763 - timing_s/adv:0.4817402958869934 - timing_s/update_actor:15.269863871857524 - timing_s/step:29.636497838422656 - timing_s/stop_profile:0.00012066960334777832 - timing_per_token_ms/adv:0.003572945901409133 - timing_per_token_ms/gen:0.1695044712345043 - timing_per_token_ms/update_actor:0.11325271728738058 - perf/total_num_tokens:134830 - perf/time_per_step:29.636497838422656 - perf/throughput:568.6822407926255 (TaskRunner pid=2026458) Training Progress: 56%|█████▌ | 56/100 [39:58<23:46, 32.42s/it] (TaskRunner pid=2026458) step:57 - global_seqlen/min:15145 - global_seqlen/max:21004 - global_seqlen/minmax_diff:5859 - global_seqlen/balanced_min:17327 - global_seqlen/balanced_max:17330 - global_seqlen/mean:17329.125 - actor/entropy:0.2190825492143631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4744122624397278 - training/rollout_probs_diff_mean:0.005394987761974335 - training/rollout_probs_diff_std:0.014672977849841118 - training/rollout_actor_probs_pearson_corr:0.9977869987487793 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.578125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:1.0002177953720093 - rollout_corr/rollout_is_ratio_fraction_high:9.200055501423776e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00013800083252135664 - rollout_corr/rollout_is_max:3.8171095848083496 - rollout_corr/rollout_is_min:0.15978100895881653 - rollout_corr/rollout_is_std:0.04392360523343086 - rollout_corr/rollout_is_eff_sample_size:0.9980752630793054 - rollout_corr/rollout_is_seq_mean:1.0002235174179077 - rollout_corr/rollout_is_seq_std:0.0027234358713030815 - rollout_corr/rollout_is_seq_max:1.0101075172424316 - rollout_corr/rollout_is_seq_min:0.9909296035766602 - rollout_corr/rollout_is_seq_max_deviation:0.01010751724243164 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2523039323277771) - rollout_corr/training_log_ppl:np.float64(0.22247063313261606) - rollout_corr/kl:np.float64(0.0011533432969415003) - rollout_corr/k3_kl:np.float64(0.0011055539546305226) - rollout_corr/rollout_ppl:np.float64(1.2508012815378606) - rollout_corr/rollout_log_ppl:np.float64(0.22131728671956807) - rollout_corr/log_ppl_diff:np.float64(0.0011533464130479842) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025634898629505187) - rollout_corr/log_ppl_diff_max:np.float64(0.016222700476646423) - rollout_corr/log_ppl_diff_min:np.float64(-0.007622018456459045) - rollout_corr/ppl_ratio:np.float64(1.0011591671500355) - rollout_corr/chi2_token:np.float64(0.0020452477037906647) - rollout_corr/chi2_seq:np.float64(0.5157537555787712) - actor/pg_loss:np.float64(0.003759522340260446) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008975071282293356) - actor/ppo_kl:np.float64(0.00035579913428307464) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.578125) - self_distillation/token_reweight_w_mean:np.float64(236077.81382388272) - self_distillation/token_reweight_w_std:np.float64(2308889.815033672) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994992383290082) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0630278438620735) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.43273352086544037) - perf/mfu/actor:np.float64(0.031510354454649704) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.82762956619263) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:0 - critic/score/mean:0.578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0027676832396537066 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0027676832396537066 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:254.75390625 - response_length/max:747.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:254.75390625 - response_length_non_aborted/max:747.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.78125 - prompt_length/max:660.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014020688831806183 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4702318627387285) - timing_s/agent_loop/generate_sequences/max:np.float64(5.407115560024977) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.896707163978135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.407115560024977) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:270 - timing_s/agent_loop/slowest/response_length:747 - timing_s/gen:11.385951856151223 - timing_s/reward:0.05382969416677952 - timing_s/old_log_prob:2.539514096453786 - timing_s/adv:0.49977708980441093 - timing_s/update_actor:15.644645780324936 - timing_s/step:30.209736336022615 - timing_s/stop_profile:0.00012044794857501984 - timing_per_token_ms/adv:0.0036050369666992053 - timing_per_token_ms/gen:0.17458564264150794 - timing_per_token_ms/update_actor:0.1128493632852563 - perf/total_num_tokens:138633 - perf/time_per_step:30.209736336022615 - perf/throughput:573.6271514338392 (TaskRunner pid=2026458) Training Progress: 57%|█████▋ | 57/100 [40:28<22:46, 31.77s/it] (TaskRunner pid=2026458) step:58 - global_seqlen/min:14734 - global_seqlen/max:19599 - global_seqlen/minmax_diff:4865 - global_seqlen/balanced_min:17102 - global_seqlen/balanced_max:17103 - global_seqlen/mean:17102.75 - actor/entropy:0.23608146607875824 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5141049027442932 - training/rollout_probs_diff_mean:0.005427653901278973 - training/rollout_probs_diff_std:0.013945525512099266 - training/rollout_actor_probs_pearson_corr:0.9981228709220886 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999080896377563 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.5163001990003977e-05 - rollout_corr/rollout_is_max:1.724382996559143 - rollout_corr/rollout_is_min:0.16556662321090698 - rollout_corr/rollout_is_std:0.04180582985281944 - rollout_corr/rollout_is_eff_sample_size:0.9982550842404107 - rollout_corr/rollout_is_seq_mean:0.9999476671218872 - rollout_corr/rollout_is_seq_std:0.002901083789765835 - rollout_corr/rollout_is_seq_max:1.00826895236969 - rollout_corr/rollout_is_seq_min:0.9917892217636108 - rollout_corr/rollout_is_seq_max_deviation:0.008268952369689941 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2862667385488749) - rollout_corr/training_log_ppl:np.float64(0.24756408171379007) - rollout_corr/kl:np.float64(0.0008958796237372724) - rollout_corr/k3_kl:np.float64(0.0010313347952433105) - rollout_corr/rollout_ppl:np.float64(1.2851230534724891) - rollout_corr/rollout_log_ppl:np.float64(0.24666820294805802) - rollout_corr/log_ppl_diff:np.float64(0.0008958787657320499) - rollout_corr/log_ppl_abs_diff:np.float64(0.002455878769978881) - rollout_corr/log_ppl_diff_max:np.float64(0.014661163091659546) - rollout_corr/log_ppl_diff_min:np.float64(-0.006324887275695801) - rollout_corr/ppl_ratio:np.float64(1.00090107973665) - rollout_corr/chi2_token:np.float64(0.0023643041495233774) - rollout_corr/chi2_seq:np.float64(0.6059110292699188) - actor/pg_loss:np.float64(0.003710375865921378) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011193532754987245) - actor/ppo_kl:np.float64(-8.923532409355062e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(286496.5454768897) - self_distillation/token_reweight_w_std:np.float64(3628499.957205932) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0005135682877153) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07967197702964768) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.5009380057454109) - perf/mfu/actor:np.float64(0.03152191694270064) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.63263320922852) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0016660349210724235 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0016660349210724235 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:257.6171875 - response_length/max:583.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:257.6171875 - response_length_non_aborted/max:583.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:598.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001420155167579651 - timing_s/agent_loop/generate_sequences/min:np.float64(1.489874416962266) - timing_s/agent_loop/generate_sequences/max:np.float64(4.795747706666589) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8081408347570687) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.795747706666589) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:583 - timing_s/gen:10.590774795040488 - timing_s/reward:0.05500875972211361 - timing_s/old_log_prob:2.199430391192436 - timing_s/adv:0.4823594633489847 - timing_s/update_actor:15.483584174886346 - timing_s/step:28.898630160838366 - timing_s/stop_profile:0.0001272875815629959 - timing_per_token_ms/adv:0.0035254525101883084 - timing_per_token_ms/gen:0.16058794230539028 - timing_per_token_ms/update_actor:0.11316589565191523 - perf/total_num_tokens:136822 - perf/time_per_step:28.898630160838366 - perf/throughput:591.818709219532 (TaskRunner pid=2026458) Training Progress: 58%|█████▊ | 58/100 [40:57<21:38, 30.92s/it] (TaskRunner pid=2026458) step:59 - global_seqlen/min:15244 - global_seqlen/max:19485 - global_seqlen/minmax_diff:4241 - global_seqlen/balanced_min:16769 - global_seqlen/balanced_max:16771 - global_seqlen/mean:16770.625 - actor/entropy:0.2315026819705963 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5046374797821045 - training/rollout_probs_diff_mean:0.0052477153949439526 - training/rollout_probs_diff_std:0.013823322020471096 - training/rollout_actor_probs_pearson_corr:0.9981176853179932 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:1.0000110864639282 - rollout_corr/rollout_is_ratio_fraction_high:1.5287481801351532e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.114992720540613e-05 - rollout_corr/rollout_is_max:2.5501413345336914 - rollout_corr/rollout_is_min:0.3053041398525238 - rollout_corr/rollout_is_std:0.04093707725405693 - rollout_corr/rollout_is_eff_sample_size:0.998326959562596 - rollout_corr/rollout_is_seq_mean:1.000074863433838 - rollout_corr/rollout_is_seq_std:0.0029152550268918276 - rollout_corr/rollout_is_seq_max:1.0087932348251343 - rollout_corr/rollout_is_seq_min:0.9920890927314758 - rollout_corr/rollout_is_seq_max_deviation:0.008793234825134277 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2814294979907572) - rollout_corr/training_log_ppl:np.float64(0.24554033184540458) - rollout_corr/kl:np.float64(0.0010128226648049576) - rollout_corr/k3_kl:np.float64(0.0010760613429283694) - rollout_corr/rollout_ppl:np.float64(1.280145457945764) - rollout_corr/rollout_log_ppl:np.float64(0.2445275079808198) - rollout_corr/log_ppl_diff:np.float64(0.0010128238645847887) - rollout_corr/log_ppl_abs_diff:np.float64(0.0026435373874846846) - rollout_corr/log_ppl_diff_max:np.float64(0.01640312373638153) - rollout_corr/log_ppl_diff_min:np.float64(-0.01078614592552185) - rollout_corr/ppl_ratio:np.float64(1.001019265735522) - rollout_corr/chi2_token:np.float64(0.0022960123606026173) - rollout_corr/chi2_seq:np.float64(0.9392846249975264) - actor/pg_loss:np.float64(0.002857705345377326) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00044364196128299227) - actor/ppo_kl:np.float64(0.0001824798405234418) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(71359.55967221712) - self_distillation/token_reweight_w_std:np.float64(989988.4983619121) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995234010275453) - self_distillation/token_reweight_w_clip_frac:np.float64(0.04141386490664445) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.3792598769068718) - perf/mfu/actor:np.float64(0.03147454843432276) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(144.65855026245117) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:0 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003848623251542449 - critic/advantages/max:0.625 - critic/advantages/min:-0.75 - critic/returns/mean:0.003848623251542449 - critic/returns/max:0.625 - critic/returns/min:-0.75 - response_length/mean:255.51953125 - response_length/max:621.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:255.51953125 - response_length_non_aborted/max:621.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.5625 - prompt_length/max:615.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.882924258708954e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.939049456268549) - timing_s/agent_loop/generate_sequences/max:np.float64(5.183893235400319) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.658139636150736) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.183893235400319) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:217 - timing_s/agent_loop/slowest/response_length:621 - timing_s/gen:10.904466819018126 - timing_s/reward:0.052795736119151115 - timing_s/old_log_prob:2.23296419903636 - timing_s/adv:0.47304221242666245 - timing_s/update_actor:15.1854842081666 - timing_s/step:28.936049316078424 - timing_s/stop_profile:5.451589822769165e-05 - timing_per_token_ms/adv:0.0035258242643510787 - timing_per_token_ms/gen:0.16670183020222473 - timing_per_token_ms/update_actor:0.11318513925514552 - perf/total_num_tokens:134165 - perf/time_per_step:28.936049316078424 - perf/throughput:579.575491346752 (TaskRunner pid=2026458) Training Progress: 59%|█████▉ | 59/100 [41:26<20:43, 30.33s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) To determine the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", we need to identify which of the given options contains the correct functional groups and structure that match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a nitrogen-containing group (n1) connected to a cyclohexane ring (Cc2ccc(Cl)cc2), with a Cl group and a trifluoromethyl group (C(F)(F)F). The correct reactant must include these key features. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1) connected to a cyclohexane ring with a Cl group and a trifluoromethyl group. This matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it does not include the Cl group or the trifluoromethyl group in the correct position. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it includes a different functional group (Oc2ccc(Cl)c(C(F)(F)F)c2) that does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D contains a ketone group (O=C1), an aromatic ring (c2ccccc2), and a nitrogen (N1), but it includes a different functional group (CC=C(Cc1ccccc1)C(F)(F)F) that does not match the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2026458) step:60 - global_seqlen/min:14552 - global_seqlen/max:17730 - global_seqlen/minmax_diff:3178 - global_seqlen/balanced_min:16889 - global_seqlen/balanced_max:16891 - global_seqlen/mean:16890.25 - actor/entropy:0.258317232131958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5851317644119263 - training/rollout_probs_diff_mean:0.005422248970717192 - training/rollout_probs_diff_std:0.013785075396299362 - training/rollout_actor_probs_pearson_corr:0.9982817769050598 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.63671875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0001024007797241 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4916022337274626e-05 - rollout_corr/rollout_is_max:1.976409912109375 - rollout_corr/rollout_is_min:0.3921801745891571 - rollout_corr/rollout_is_std:0.041325442492961884 - rollout_corr/rollout_is_eff_sample_size:0.9982955945128663 - rollout_corr/rollout_is_seq_mean:1.000079870223999 - rollout_corr/rollout_is_seq_std:0.002677295822650194 - rollout_corr/rollout_is_seq_max:1.0086641311645508 - rollout_corr/rollout_is_seq_min:0.9929971694946289 - rollout_corr/rollout_is_seq_max_deviation:0.008664131164550781 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3128179563209414) - rollout_corr/training_log_ppl:np.float64(0.2678291550837457) - rollout_corr/kl:np.float64(0.001152293861238718) - rollout_corr/k3_kl:np.float64(0.0010784027733734547) - rollout_corr/rollout_ppl:np.float64(1.3112971791997552) - rollout_corr/rollout_log_ppl:np.float64(0.2666768589988351) - rollout_corr/log_ppl_diff:np.float64(0.0011522960849106312) - rollout_corr/log_ppl_abs_diff:np.float64(0.002510662190616131) - rollout_corr/log_ppl_diff_max:np.float64(0.009669288992881775) - rollout_corr/log_ppl_diff_min:np.float64(-0.007052525877952576) - rollout_corr/ppl_ratio:np.float64(1.0011574435047805) - rollout_corr/chi2_token:np.float64(0.002051107119768858) - rollout_corr/chi2_seq:np.float64(0.5519429405685514) - actor/pg_loss:np.float64(0.003786680055782199) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008521524032403249) - actor/ppo_kl:np.float64(0.0003417033792416646) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.63671875) - self_distillation/token_reweight_w_mean:np.float64(148988.88128135214) - self_distillation/token_reweight_w_std:np.float64(1866554.9818183868) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014604937750846) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08368598294327967) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.4603109806776047) - perf/mfu/actor:np.float64(0.03142972200463806) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.4005994796753) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/reward/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07287959815596419) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8152857142857142) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.0635348044707675) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6967476190476191) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.04617062732616158) - val-aux/sciknoweval/score/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/score/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07287959815596419) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8152857142857142) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.0635348044707675) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6967476190476191) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.04617062732616158) - val-core/sciknoweval/acc/mean@16:np.float64(0.6735119047619048) - val-aux/sciknoweval/acc/std@16:np.float64(0.14333415818798717) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7287000000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10654141009401605) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6172761904761904) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.13505886263851702) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6726047619047619) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1435861836456534) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7666619047619049) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0728423418058757) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5634142857142856) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11842617318668035) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6849761904761906) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10836775791233498) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7947571428571428) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05056443439799785) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5141142857142857) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09350537912002374) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6924571428571429) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07287959815596419) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8152857142857142) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03269928922744704) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.47582857142857143) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.0635348044707675) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6967476190476191) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.04617062732616158) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:1 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.000639524485450238 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.000639524485450238 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:261.8828125 - response_length/max:611.0 - response_length/min:121.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:261.8828125 - response_length_non_aborted/max:611.0 - response_length_non_aborted/min:121.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.9375 - prompt_length/max:451.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00021154806017875671 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3727056793868542) - timing_s/agent_loop/generate_sequences/max:np.float64(4.906513184309006) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9096029985012137) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.906513184309006) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:563 - timing_s/gen:11.531464690342546 - timing_s/reward:0.05282927677035332 - timing_s/old_log_prob:2.459550989791751 - timing_s/adv:0.5421040039509535 - timing_s/update_actor:15.288631148636341 - timing_s/step:29.9776940792799 - timing_s/testing:64.2218036558479 - timing_s/save_checkpoint:6.659373983740807 - timing_s/stop_profile:5.859695374965668e-05 - timing_per_token_ms/adv:0.00401195959170937 - timing_per_token_ms/gen:0.17200359014263514 - timing_per_token_ms/update_actor:0.11314686837551502 - perf/total_num_tokens:135122 - perf/time_per_step:29.9776940792799 - perf/throughput:563.4272587922054 (TaskRunner pid=2026458) Training Progress: 60%|██████ | 60/100 [43:07<34:24, 51.61s/it] (TaskRunner pid=2026458) step:61 - global_seqlen/min:14736 - global_seqlen/max:19497 - global_seqlen/minmax_diff:4761 - global_seqlen/balanced_min:17532 - global_seqlen/balanced_max:17536 - global_seqlen/mean:17534.5 - actor/entropy:0.2423284649848938 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.49669769406318665 - training/rollout_probs_diff_mean:0.005022233817726374 - training/rollout_probs_diff_std:0.013324090279638767 - training/rollout_actor_probs_pearson_corr:0.9983177781105042 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.86328125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.86328125 - rollout_corr/rollout_is_mean:1.0002062320709229 - rollout_corr/rollout_is_ratio_fraction_high:5.464779314934276e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0985847590491176e-05 - rollout_corr/rollout_is_max:7.0302934646606445 - rollout_corr/rollout_is_min:0.35777419805526733 - rollout_corr/rollout_is_std:0.03978368639945984 - rollout_corr/rollout_is_eff_sample_size:0.998420353539922 - rollout_corr/rollout_is_seq_mean:1.0001554489135742 - rollout_corr/rollout_is_seq_std:0.002474896376952529 - rollout_corr/rollout_is_seq_max:1.0129971504211426 - rollout_corr/rollout_is_seq_min:0.9919136762619019 - rollout_corr/rollout_is_seq_max_deviation:0.012997150421142578 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2963423300534487) - rollout_corr/training_log_ppl:np.float64(0.25556481117382646) - rollout_corr/kl:np.float64(0.0009956922927543133) - rollout_corr/k3_kl:np.float64(0.0010460353189500893) - rollout_corr/rollout_ppl:np.float64(1.2950277719646692) - rollout_corr/rollout_log_ppl:np.float64(0.25456911721266806) - rollout_corr/log_ppl_diff:np.float64(0.0009956939611583948) - rollout_corr/log_ppl_abs_diff:np.float64(0.002202652278356254) - rollout_corr/log_ppl_diff_max:np.float64(0.010044902563095093) - rollout_corr/log_ppl_diff_min:np.float64(-0.009525060653686523) - rollout_corr/ppl_ratio:np.float64(1.0009999251924455) - rollout_corr/chi2_token:np.float64(0.0022548860870301723) - rollout_corr/chi2_seq:np.float64(0.8442793434951454) - actor/pg_loss:np.float64(0.0042004447896033525) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007894714767644473) - actor/ppo_kl:np.float64(0.0002806229531095994) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.86328125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.86328125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(132276.08868701942) - self_distillation/token_reweight_w_std:np.float64(1859101.6540320972) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012411118950695) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08409801000379957) - self_distillation/empty_target_batch:np.float64(0.13671875) - actor/grad_norm:np.float64(0.4940867945551872) - perf/mfu/actor:np.float64(0.03230972888329671) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(146.0987777709961) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0024950134102255106 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0024950134102255106 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:285.921875 - response_length/max:685.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:285.921875 - response_length_non_aborted/max:685.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.03125 - prompt_length/max:462.0 - prompt_length/min:177.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.605738937854767e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.467216296121478) - timing_s/agent_loop/generate_sequences/max:np.float64(5.52247816324234) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0488981239686836) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.52247816324234) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:202 - timing_s/agent_loop/slowest/response_length:652 - timing_s/gen:11.610483139753342 - timing_s/reward:0.05246041156351566 - timing_s/old_log_prob:2.416056863963604 - timing_s/adv:0.5235678162425756 - timing_s/update_actor:15.423095140606165 - timing_s/step:30.12737883068621 - timing_s/stop_profile:0.00011511333286762238 - timing_per_token_ms/adv:0.003732411932494337 - timing_per_token_ms/gen:0.15862182550622086 - timing_per_token_ms/update_actor:0.10994821024698569 - perf/total_num_tokens:140276 - perf/time_per_step:30.12737883068621 - perf/throughput:582.0121325038822 (TaskRunner pid=2026458) Training Progress: 61%|██████ | 61/100 [43:37<29:22, 45.18s/it] (TaskRunner pid=2026458) step:62 - global_seqlen/min:15907 - global_seqlen/max:20488 - global_seqlen/minmax_diff:4581 - global_seqlen/balanced_min:18196 - global_seqlen/balanced_max:18198 - global_seqlen/mean:18197.0 - actor/entropy:0.26120486855506897 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35323476791381836 - training/rollout_probs_diff_mean:0.005538288038223982 - training/rollout_probs_diff_std:0.014037296175956726 - training/rollout_actor_probs_pearson_corr:0.9981968998908997 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.64453125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.9998036623001099 - rollout_corr/rollout_is_ratio_fraction_high:2.8998956622672267e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014499477401841432 - rollout_corr/rollout_is_max:2.314438819885254 - rollout_corr/rollout_is_min:0.32504311203956604 - rollout_corr/rollout_is_std:0.04294585436582565 - rollout_corr/rollout_is_eff_sample_size:0.9981582768191568 - rollout_corr/rollout_is_seq_mean:0.9998564720153809 - rollout_corr/rollout_is_seq_std:0.0027946694754064083 - rollout_corr/rollout_is_seq_max:1.0084697008132935 - rollout_corr/rollout_is_seq_min:0.9910843372344971 - rollout_corr/rollout_is_seq_max_deviation:0.00891566276550293 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.316816312726587) - rollout_corr/training_log_ppl:np.float64(0.27135412095231004) - rollout_corr/kl:np.float64(0.0013785573038145316) - rollout_corr/k3_kl:np.float64(0.0011328787384172756) - rollout_corr/rollout_ppl:np.float64(1.3149180496111512) - rollout_corr/rollout_log_ppl:np.float64(0.2699755621724762) - rollout_corr/log_ppl_diff:np.float64(0.0013785587798338383) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027030363271478564) - rollout_corr/log_ppl_diff_max:np.float64(0.010782003402709961) - rollout_corr/log_ppl_diff_min:np.float64(-0.008555859327316284) - rollout_corr/ppl_ratio:np.float64(1.0013843621127307) - rollout_corr/chi2_token:np.float64(0.0016965712420642376) - rollout_corr/chi2_seq:np.float64(0.6767998114228249) - actor/pg_loss:np.float64(0.003519455436617136) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008139853748616588) - actor/ppo_kl:np.float64(0.0002313825499626887) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.64453125) - self_distillation/token_reweight_w_mean:np.float64(140065.4251762007) - self_distillation/token_reweight_w_std:np.float64(1931525.736894751) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999061315320432) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07281633446109481) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.4214628040790558) - perf/mfu/actor:np.float64(0.033835202551722315) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(146.0380401611328) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:1 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003894922323524952 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003894922323524952 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:269.40625 - response_length/max:578.0 - response_length/min:124.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:269.40625 - response_length_non_aborted/max:578.0 - response_length_non_aborted/min:124.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.25 - prompt_length/max:662.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001499168574810028 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3069061245769262) - timing_s/agent_loop/generate_sequences/max:np.float64(4.653931180015206) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8548159639394726) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.653931180015206) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:578 - timing_s/gen:10.786067547276616 - timing_s/reward:0.05438114143908024 - timing_s/old_log_prob:2.2739491928368807 - timing_s/adv:0.471931716427207 - timing_s/update_actor:15.285212887451053 - timing_s/step:28.96465684659779 - timing_s/stop_profile:0.00012806802988052368 - timing_per_token_ms/adv:0.003241823627707912 - timing_per_token_ms/gen:0.15639234931093574 - timing_per_token_ms/update_actor:0.10499816513333965 - perf/total_num_tokens:145576 - perf/time_per_step:28.96465684659779 - perf/throughput:628.2484234622456 (TaskRunner pid=2026458) Training Progress: 62%|██████▏ | 62/100 [44:06<25:32, 40.33s/it] (TaskRunner pid=2026458) step:63 - global_seqlen/min:15346 - global_seqlen/max:20074 - global_seqlen/minmax_diff:4728 - global_seqlen/balanced_min:17670 - global_seqlen/balanced_max:17672 - global_seqlen/mean:17671.0 - actor/entropy:0.2494240403175354 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44295963644981384 - training/rollout_probs_diff_mean:0.005514265038073063 - training/rollout_probs_diff_std:0.013980090618133545 - training/rollout_actor_probs_pearson_corr:0.9981726408004761 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.48828125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:0.9999319911003113 - rollout_corr/rollout_is_ratio_fraction_high:1.4685150745208375e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.811090083327144e-05 - rollout_corr/rollout_is_max:2.1109066009521484 - rollout_corr/rollout_is_min:0.21203994750976562 - rollout_corr/rollout_is_std:0.04199644550681114 - rollout_corr/rollout_is_eff_sample_size:0.9982390473577023 - rollout_corr/rollout_is_seq_mean:0.9999494552612305 - rollout_corr/rollout_is_seq_std:0.003092808648943901 - rollout_corr/rollout_is_seq_max:1.0130763053894043 - rollout_corr/rollout_is_seq_min:0.9914883375167847 - rollout_corr/rollout_is_seq_max_deviation:0.013076305389404297 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.302847915329039) - rollout_corr/training_log_ppl:np.float64(0.26041782830725424) - rollout_corr/kl:np.float64(0.001262190604840896) - rollout_corr/k3_kl:np.float64(0.0011403532549252304) - rollout_corr/rollout_ppl:np.float64(1.3011768972501159) - rollout_corr/rollout_log_ppl:np.float64(0.2591556366533041) - rollout_corr/log_ppl_diff:np.float64(0.0012621916539501399) - rollout_corr/log_ppl_abs_diff:np.float64(0.002628726389957592) - rollout_corr/log_ppl_diff_max:np.float64(0.010536238551139832) - rollout_corr/log_ppl_diff_min:np.float64(-0.009842857718467712) - rollout_corr/ppl_ratio:np.float64(1.0012680946383625) - rollout_corr/chi2_token:np.float64(0.0019653229974210262) - rollout_corr/chi2_seq:np.float64(0.7706008821260184) - actor/pg_loss:np.float64(0.0034944345243275166) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000709521887529263) - actor/ppo_kl:np.float64(0.00027179806370725146) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.48828125) - self_distillation/token_reweight_w_mean:np.float64(168796.3436012969) - self_distillation/token_reweight_w_std:np.float64(2124951.2078802614) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015958338044584) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0751116150058806) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.4171854108572006) - perf/mfu/actor:np.float64(0.03263190662040404) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.89733123779297) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:1 - critic/score/mean:0.48828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.48828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005943814758211374 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005943814758211374 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:266.0 - response_length/max:610.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:266.0 - response_length_non_aborted/max:610.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.21875 - prompt_length/max:532.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011231563985347748 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2275551408529282) - timing_s/agent_loop/generate_sequences/max:np.float64(4.99046435020864) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8069054595180205) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.99046435020864) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:231 - timing_s/agent_loop/slowest/response_length:607 - timing_s/gen:11.166345795616508 - timing_s/reward:0.05529453977942467 - timing_s/old_log_prob:2.2847698368132114 - timing_s/adv:0.49184093065559864 - timing_s/update_actor:15.462059672921896 - timing_s/step:29.548018461093307 - timing_s/stop_profile:0.00011456944048404694 - timing_per_token_ms/adv:0.0034791532076254785 - timing_per_token_ms/gen:0.1639794671583721 - timing_per_token_ms/update_actor:0.10937453789345464 - perf/total_num_tokens:141368 - perf/time_per_step:29.548018461093307 - perf/throughput:598.0434871890951 (TaskRunner pid=2026458) Training Progress: 63%|██████▎ | 63/100 [44:36<22:53, 37.11s/it] (TaskRunner pid=2026458) step:64 - global_seqlen/min:14067 - global_seqlen/max:24197 - global_seqlen/minmax_diff:10130 - global_seqlen/balanced_min:18752 - global_seqlen/balanced_max:25893 - global_seqlen/mean:19646.75 - actor/entropy:0.21532860398292542 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8697976469993591 - training/rollout_probs_diff_mean:0.004864232614636421 - training/rollout_probs_diff_std:0.013850908726453781 - training/rollout_actor_probs_pearson_corr:0.9980413317680359 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.70703125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:1.0002894401550293 - rollout_corr/rollout_is_ratio_fraction_high:6.0805057728430256e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.4322023818967864e-05 - rollout_corr/rollout_is_max:8.297144889831543 - rollout_corr/rollout_is_min:0.23644955456256866 - rollout_corr/rollout_is_std:0.04067561775445938 - rollout_corr/rollout_is_eff_sample_size:0.9983489400333069 - rollout_corr/rollout_is_seq_mean:1.000409722328186 - rollout_corr/rollout_is_seq_std:0.0031241707038134336 - rollout_corr/rollout_is_seq_max:1.0219839811325073 - rollout_corr/rollout_is_seq_min:0.9937897324562073 - rollout_corr/rollout_is_seq_max_deviation:0.021983981132507324 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2946171956136823) - rollout_corr/training_log_ppl:np.float64(0.25466993783356884) - rollout_corr/kl:np.float64(0.0008532866019912433) - rollout_corr/k3_kl:np.float64(0.0010788948769118178) - rollout_corr/rollout_ppl:np.float64(1.2935234676115215) - rollout_corr/rollout_log_ppl:np.float64(0.25381665135137155) - rollout_corr/log_ppl_diff:np.float64(0.0008532864821972908) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022936501272852183) - rollout_corr/log_ppl_diff_max:np.float64(0.00881144404411316) - rollout_corr/log_ppl_diff_min:np.float64(-0.008881300687789917) - rollout_corr/ppl_ratio:np.float64(1.0008577948901802) - rollout_corr/chi2_token:np.float64(0.003156282240524888) - rollout_corr/chi2_seq:np.float64(3.035388365620747) - actor/pg_loss:np.float64(0.0014677918516099453) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004414845861901995) - actor/ppo_kl:np.float64(0.00026685018261352056) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_w_mean:np.float64(81162.51946377661) - self_distillation/token_reweight_w_std:np.float64(1170164.887981981) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010531947482377) - self_distillation/token_reweight_w_clip_frac:np.float64(0.049339942750521004) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.263648372143507) - perf/mfu/actor:np.float64(0.03462126720848994) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.74031829833984) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:1 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.08103185892105103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.08103185892105103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:321.2109375 - response_length/max:8192.0 - response_length/min:110.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:321.2109375 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:292.75 - prompt_length/max:495.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010729953646659851 - timing_s/agent_loop/generate_sequences/min:np.float64(1.288361655548215) - timing_s/agent_loop/generate_sequences/max:np.float64(50.12881679646671) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2828551806887845) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.12881679646671) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:172 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.03219731338322 - timing_s/reward:0.05905804969370365 - timing_s/old_log_prob:2.5028711035847664 - timing_s/adv:0.508072393015027 - timing_s/update_actor:16.261700792238116 - timing_s/step:75.45720112323761 - timing_s/stop_profile:4.878826439380646e-05 - timing_per_token_ms/adv:0.0032325473234442533 - timing_per_token_ms/gen:0.6814082124940195 - timing_per_token_ms/update_actor:0.10346304600148953 - perf/total_num_tokens:157174 - perf/time_per_step:75.45720112323761 - perf/throughput:260.3694505964075 (TaskRunner pid=2026458) Training Progress: 64%|██████▍ | 64/100 [45:51<29:10, 48.62s/it] (TaskRunner pid=2026458) step:65 - global_seqlen/min:15781 - global_seqlen/max:21398 - global_seqlen/minmax_diff:5617 - global_seqlen/balanced_min:18710 - global_seqlen/balanced_max:18715 - global_seqlen/mean:18712.625 - actor/entropy:0.2451825588941574 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43764662742614746 - training/rollout_probs_diff_mean:0.005050553474575281 - training/rollout_probs_diff_std:0.013090502470731735 - training/rollout_actor_probs_pearson_corr:0.9983762502670288 - self_distillation/success_group_fraction:0.875 - self_distillation/success_sample_fraction:0.8671875 - self_distillation/correct_sample_fraction:0.74609375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8671875 - rollout_corr/rollout_is_mean:0.9999321103096008 - rollout_corr/rollout_is_ratio_fraction_high:1.2970000170753337e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.8910002331249416e-05 - rollout_corr/rollout_is_max:2.924093246459961 - rollout_corr/rollout_is_min:0.4824545979499817 - rollout_corr/rollout_is_std:0.040282461792230606 - rollout_corr/rollout_is_eff_sample_size:0.9983795954951445 - rollout_corr/rollout_is_seq_mean:0.9999394416809082 - rollout_corr/rollout_is_seq_std:0.002558288164436817 - rollout_corr/rollout_is_seq_max:1.0071020126342773 - rollout_corr/rollout_is_seq_min:0.9918232560157776 - rollout_corr/rollout_is_seq_max_deviation:0.008176743984222412 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2973304083570838) - rollout_corr/training_log_ppl:np.float64(0.2564336106006522) - rollout_corr/kl:np.float64(0.0012040652124341022) - rollout_corr/k3_kl:np.float64(0.000990912636439134) - rollout_corr/rollout_ppl:np.float64(1.2957068742252886) - rollout_corr/rollout_log_ppl:np.float64(0.2552295460482128) - rollout_corr/log_ppl_diff:np.float64(0.0012040645524393767) - rollout_corr/log_ppl_abs_diff:np.float64(0.002526016061892733) - rollout_corr/log_ppl_diff_max:np.float64(0.011661797761917114) - rollout_corr/log_ppl_diff_min:np.float64(-0.006277412176132202) - rollout_corr/ppl_ratio:np.float64(1.001209291163832) - rollout_corr/chi2_token:np.float64(0.0015423318836838007) - rollout_corr/chi2_seq:np.float64(0.9701050808653235) - actor/pg_loss:np.float64(0.0032792859710752964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007688776604481973) - actor/ppo_kl:np.float64(0.0002768948086071532) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8671875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8671875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_w_mean:np.float64(86791.39557419321) - self_distillation/token_reweight_w_std:np.float64(1084418.8798323106) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9982076121959835) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08052845095517114) - self_distillation/empty_target_batch:np.float64(0.1328125) - actor/grad_norm:np.float64(0.3758349269628525) - perf/mfu/actor:np.float64(0.03501136292112507) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.70735931396484) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:1 - critic/score/mean:0.74609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.74609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0052852751687169075 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0052852751687169075 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.17578125 - response_length/max:840.0 - response_length/min:129.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.17578125 - response_length_non_aborted/max:840.0 - response_length_non_aborted/min:129.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:283.59375 - prompt_length/max:818.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.040077328681946e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2666476387530565) - timing_s/agent_loop/generate_sequences/max:np.float64(6.11080476641655) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.148740480595734) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.11080476641655) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:228 - timing_s/agent_loop/slowest/response_length:840 - timing_s/gen:12.000718062743545 - timing_s/reward:0.056409286335110664 - timing_s/old_log_prob:2.2715286798775196 - timing_s/adv:0.546351995319128 - timing_s/update_actor:15.278582841157913 - timing_s/step:30.242278227582574 - timing_s/stop_profile:0.0001621972769498825 - timing_per_token_ms/adv:0.0036496215477460274 - timing_per_token_ms/gen:0.1556493179432633 - timing_per_token_ms/update_actor:0.10206065985636645 - perf/total_num_tokens:149701 - perf/time_per_step:30.242278227582574 - perf/throughput:618.757120716292 (TaskRunner pid=2026458) Training Progress: 65%|██████▌ | 65/100 [46:22<25:09, 43.12s/it] (TaskRunner pid=2026458) step:66 - global_seqlen/min:16986 - global_seqlen/max:21356 - global_seqlen/minmax_diff:4370 - global_seqlen/balanced_min:19034 - global_seqlen/balanced_max:19038 - global_seqlen/mean:19037.375 - actor/entropy:0.2407594919204712 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.26579251885414124 - training/rollout_probs_diff_mean:0.005038884002715349 - training/rollout_probs_diff_std:0.013142738491296768 - training/rollout_actor_probs_pearson_corr:0.9983550906181335 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:1.000045657157898 - rollout_corr/rollout_is_ratio_fraction_high:2.5642670152592473e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.974934462457895e-05 - rollout_corr/rollout_is_max:2.355177402496338 - rollout_corr/rollout_is_min:0.19801293313503265 - rollout_corr/rollout_is_std:0.04063016548752785 - rollout_corr/rollout_is_eff_sample_size:0.9983520292581156 - rollout_corr/rollout_is_seq_mean:1.0001461505889893 - rollout_corr/rollout_is_seq_std:0.0026480352971702814 - rollout_corr/rollout_is_seq_max:1.0091423988342285 - rollout_corr/rollout_is_seq_min:0.9913557171821594 - rollout_corr/rollout_is_seq_max_deviation:0.009142398834228516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3078779838979244) - rollout_corr/training_log_ppl:np.float64(0.26354357780655846) - rollout_corr/kl:np.float64(0.0010749662497779866) - rollout_corr/k3_kl:np.float64(0.0014432408553943787) - rollout_corr/rollout_ppl:np.float64(1.3064235104247928) - rollout_corr/rollout_log_ppl:np.float64(0.2624686111230403) - rollout_corr/log_ppl_diff:np.float64(0.0010749666835181415) - rollout_corr/log_ppl_abs_diff:np.float64(0.0029108942835591733) - rollout_corr/log_ppl_diff_max:np.float64(0.013920098543167114) - rollout_corr/log_ppl_diff_min:np.float64(-0.009019121527671814) - rollout_corr/ppl_ratio:np.float64(1.0010822340846062) - rollout_corr/chi2_token:np.float64(0.003545147832483053) - rollout_corr/chi2_seq:np.float64(1.548627549316734) - actor/pg_loss:np.float64(0.003003176534548402) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009145650715254305) - actor/ppo_kl:np.float64(0.0003188287759954278) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.546875) - self_distillation/token_reweight_w_mean:np.float64(126959.81397781824) - self_distillation/token_reweight_w_std:np.float64(1629415.2238851967) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0008254635613412) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06369021107093431) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.4391012266278267) - perf/mfu/actor:np.float64(0.03551272812953314) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.41056060791016) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004540354944765568 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004540354944765568 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:304.66796875 - response_length/max:841.0 - response_length/min:130.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:304.66796875 - response_length_non_aborted/max:841.0 - response_length_non_aborted/min:130.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.25 - prompt_length/max:617.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014583580195903778 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5362531188875437) - timing_s/agent_loop/generate_sequences/max:np.float64(6.417147822678089) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.267450157953135) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.417147822678089) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:841 - timing_s/gen:12.298093724995852 - timing_s/reward:0.05658415146172047 - timing_s/old_log_prob:2.3751799557358027 - timing_s/adv:0.4913473054766655 - timing_s/update_actor:15.358580730855465 - timing_s/step:30.66549047641456 - timing_s/stop_profile:0.0001288391649723053 - timing_per_token_ms/adv:0.0032262017838374873 - timing_per_token_ms/gen:0.1576779758317309 - timing_per_token_ms/update_actor:0.10084492170569383 - perf/total_num_tokens:152299 - perf/time_per_step:30.66549047641456 - perf/throughput:620.8077778714162 (TaskRunner pid=2026458) Training Progress: 66%|██████▌ | 66/100 [46:52<22:19, 39.40s/it] (TaskRunner pid=2026458) step:67 - global_seqlen/min:15159 - global_seqlen/max:19915 - global_seqlen/minmax_diff:4756 - global_seqlen/balanced_min:18046 - global_seqlen/balanced_max:18050 - global_seqlen/mean:18048.5 - actor/entropy:0.22801370918750763 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9852267503738403 - training/rollout_probs_diff_mean:0.005018836818635464 - training/rollout_probs_diff_std:0.014277998358011246 - training/rollout_actor_probs_pearson_corr:0.9979830384254456 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.66796875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:1.0002268552780151 - rollout_corr/rollout_is_ratio_fraction_high:2.8704287615255453e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001004650112008676 - rollout_corr/rollout_is_max:2.3428878784179688 - rollout_corr/rollout_is_min:0.0007197936065495014 - rollout_corr/rollout_is_std:0.040458161383867264 - rollout_corr/rollout_is_eff_sample_size:0.9983666439269478 - rollout_corr/rollout_is_seq_mean:1.0003077983856201 - rollout_corr/rollout_is_seq_std:0.0026670496445149183 - rollout_corr/rollout_is_seq_max:1.0164320468902588 - rollout_corr/rollout_is_seq_min:0.9921717047691345 - rollout_corr/rollout_is_seq_max_deviation:0.01643204689025879 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2767656035721302) - rollout_corr/training_log_ppl:np.float64(0.24004710461304057) - rollout_corr/kl:np.float64(0.0011609104617287258) - rollout_corr/k3_kl:np.float64(0.001655576513314827) - rollout_corr/rollout_ppl:np.float64(1.2751760962419212) - rollout_corr/rollout_log_ppl:np.float64(0.23888619328499772) - rollout_corr/log_ppl_diff:np.float64(0.0011609113280428573) - rollout_corr/log_ppl_abs_diff:np.float64(0.003236807693610899) - rollout_corr/log_ppl_diff_max:np.float64(0.017100274562835693) - rollout_corr/log_ppl_diff_min:np.float64(-0.02350728213787079) - rollout_corr/ppl_ratio:np.float64(1.0011714671272784) - rollout_corr/chi2_token:np.float64(0.004160782555118203) - rollout_corr/chi2_seq:np.float64(5.24587120115757) - actor/pg_loss:np.float64(0.0036287190159782767) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001126493662923167) - actor/ppo_kl:np.float64(0.0005246336532138685) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.66796875) - self_distillation/token_reweight_w_mean:np.float64(249329.88290784857) - self_distillation/token_reweight_w_std:np.float64(2866482.3437989554) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9994784640148282) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06014655635226518) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.471420306712389) - perf/mfu/actor:np.float64(0.033855896624554245) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.65926361083984) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:1 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004662652965635061 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004662652965635061 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:272.171875 - response_length/max:628.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.171875 - response_length_non_aborted/max:628.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.84375 - prompt_length/max:576.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.801262497901917e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2853057831525803) - timing_s/agent_loop/generate_sequences/max:np.float64(5.174439303576946) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.863578474461974) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.174439303576946) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:351 - timing_s/agent_loop/slowest/response_length:628 - timing_s/gen:10.778497187420726 - timing_s/reward:0.052504874765872955 - timing_s/old_log_prob:2.3092593122273684 - timing_s/adv:0.5003343336284161 - timing_s/update_actor:15.202262729406357 - timing_s/step:28.936630526557565 - timing_s/stop_profile:0.00012007169425487518 - timing_per_token_ms/adv:0.0034652071753083088 - timing_per_token_ms/gen:0.1546945460046605 - timing_per_token_ms/update_actor:0.10528757742614592 - perf/total_num_tokens:144388 - perf/time_per_step:28.936630526557565 - perf/throughput:623.7250043136634 (TaskRunner pid=2026458) Training Progress: 67%|██████▋ | 67/100 [47:21<19:57, 36.28s/it] (TaskRunner pid=2026458) step:68 - global_seqlen/min:14735 - global_seqlen/max:19627 - global_seqlen/minmax_diff:4892 - global_seqlen/balanced_min:17371 - global_seqlen/balanced_max:17375 - global_seqlen/mean:17373.75 - actor/entropy:0.26004546880722046 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23114943504333496 - training/rollout_probs_diff_mean:0.005509829614311457 - training/rollout_probs_diff_std:0.013461560010910034 - training/rollout_actor_probs_pearson_corr:0.9983240365982056 - self_distillation/success_group_fraction:0.59375 - self_distillation/success_sample_fraction:0.58984375 - self_distillation/correct_sample_fraction:0.4765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.58984375 - rollout_corr/rollout_is_mean:1.0001424551010132 - rollout_corr/rollout_is_ratio_fraction_high:3.430884680710733e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.14632738486398e-05 - rollout_corr/rollout_is_max:2.094249725341797 - rollout_corr/rollout_is_min:0.4616723358631134 - rollout_corr/rollout_is_std:0.04154840111732483 - rollout_corr/rollout_is_eff_sample_size:0.998277299143981 - rollout_corr/rollout_is_seq_mean:1.0000828504562378 - rollout_corr/rollout_is_seq_std:0.0031394551042467356 - rollout_corr/rollout_is_seq_max:1.0125868320465088 - rollout_corr/rollout_is_seq_min:0.9910873770713806 - rollout_corr/rollout_is_seq_max_deviation:0.012586832046508789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3149044853635132) - rollout_corr/training_log_ppl:np.float64(0.26928867847891524) - rollout_corr/kl:np.float64(0.0007745910923624422) - rollout_corr/k3_kl:np.float64(0.0011792316492460486) - rollout_corr/rollout_ppl:np.float64(1.3139065788127482) - rollout_corr/rollout_log_ppl:np.float64(0.26851408526999876) - rollout_corr/log_ppl_diff:np.float64(0.0007745932089164853) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025513150612823665) - rollout_corr/log_ppl_diff_max:np.float64(0.009900659322738647) - rollout_corr/log_ppl_diff_min:np.float64(-0.015614360570907593) - rollout_corr/ppl_ratio:np.float64(1.0007805349305272) - rollout_corr/chi2_token:np.float64(0.0031827117782086134) - rollout_corr/chi2_seq:np.float64(0.9155827462673187) - actor/pg_loss:np.float64(0.0037549110129475594) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007232226507767336) - actor/ppo_kl:np.float64(-4.5092836657545377e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.58984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.58984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.4765625) - self_distillation/token_reweight_w_mean:np.float64(174603.2610843731) - self_distillation/token_reweight_w_std:np.float64(2132750.0757954237) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9986913346219808) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0688194558606483) - self_distillation/empty_target_batch:np.float64(0.41015625) - actor/grad_norm:np.float64(0.4323424845933914) - perf/mfu/actor:np.float64(0.03227397909516918) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.99407958984375) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:1 - critic/score/mean:0.4765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.4765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.006370724178850651 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.006370724178850651 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:227.7109375 - response_length/max:542.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:227.7109375 - response_length_non_aborted/max:542.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:315.21875 - prompt_length/max:704.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010382384061813354 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2069221679121256) - timing_s/agent_loop/generate_sequences/max:np.float64(4.399640990421176) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.518257544754306) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.399640990421176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:508 - timing_s/agent_loop/slowest/response_length:542 - timing_s/gen:10.388320125639439 - timing_s/reward:0.053288890048861504 - timing_s/old_log_prob:2.246592290699482 - timing_s/adv:0.4770530089735985 - timing_s/update_actor:15.537012735381722 - timing_s/step:28.787579828873277 - timing_s/stop_profile:0.00013300031423568726 - timing_per_token_ms/adv:0.003432282962613127 - timing_per_token_ms/gen:0.1782056493916945 - timing_per_token_ms/update_actor:0.1117851121331155 - perf/total_num_tokens:138990 - perf/time_per_step:28.787579828873277 - perf/throughput:603.5154779692363 (TaskRunner pid=2026458) Training Progress: 68%|██████▊ | 68/100 [47:50<18:09, 34.04s/it] (TaskRunner pid=2026458) step:69 - global_seqlen/min:15072 - global_seqlen/max:24554 - global_seqlen/minmax_diff:9482 - global_seqlen/balanced_min:18606 - global_seqlen/balanced_max:18614 - global_seqlen/mean:18608.375 - actor/entropy:0.24838018417358398 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21835267543792725 - training/rollout_probs_diff_mean:0.004963765386492014 - training/rollout_probs_diff_std:0.012902619317173958 - training/rollout_actor_probs_pearson_corr:0.9984732270240784 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.56640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9996421337127686 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.750844502472319e-05 - rollout_corr/rollout_is_max:1.9659780263900757 - rollout_corr/rollout_is_min:0.26981014013290405 - rollout_corr/rollout_is_std:0.039976488798856735 - rollout_corr/rollout_is_eff_sample_size:0.9984033607144375 - rollout_corr/rollout_is_seq_mean:0.9996398687362671 - rollout_corr/rollout_is_seq_std:0.0026584318839013577 - rollout_corr/rollout_is_seq_max:1.0071197748184204 - rollout_corr/rollout_is_seq_min:0.9907388687133789 - rollout_corr/rollout_is_seq_max_deviation:0.009261131286621094 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3142020958475769) - rollout_corr/training_log_ppl:np.float64(0.2681919914903119) - rollout_corr/kl:np.float64(0.0011005508938133346) - rollout_corr/k3_kl:np.float64(0.0010078800825397138) - rollout_corr/rollout_ppl:np.float64(1.3127118106931448) - rollout_corr/rollout_log_ppl:np.float64(0.2670914397749584) - rollout_corr/log_ppl_diff:np.float64(0.0011005517153535038) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023552981147076935) - rollout_corr/log_ppl_diff_max:np.float64(0.013139069080352783) - rollout_corr/log_ppl_diff_min:np.float64(-0.007561713457107544) - rollout_corr/ppl_ratio:np.float64(1.001105590723455) - rollout_corr/chi2_token:np.float64(0.0017845977563410997) - rollout_corr/chi2_seq:np.float64(0.37655540159903467) - actor/pg_loss:np.float64(0.004595633479766548) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005241842873147107) - actor/ppo_kl:np.float64(-0.0001359343073588093) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.56640625) - self_distillation/token_reweight_w_mean:np.float64(242376.49935866683) - self_distillation/token_reweight_w_std:np.float64(3279716.472702818) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0019234952051193) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08817204824299552) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.47684382647275925) - perf/mfu/actor:np.float64(0.03472723156176391) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.89558792114258) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0034019839949905872 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0034019839949905872 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:271.69921875 - response_length/max:979.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:271.69921875 - response_length_non_aborted/max:979.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:309.8125 - prompt_length/max:597.0 - prompt_length/min:168.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001641698181629181 - timing_s/agent_loop/generate_sequences/min:np.float64(1.224152859300375) - timing_s/agent_loop/generate_sequences/max:np.float64(6.517763238400221) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8646460080635734) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.517763238400221) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:979 - timing_s/gen:12.344549676403403 - timing_s/reward:0.0557593759149313 - timing_s/old_log_prob:2.2437479849904776 - timing_s/adv:0.49268065579235554 - timing_s/update_actor:14.758547224104404 - timing_s/step:29.983076663687825 - timing_s/stop_profile:0.00012305378913879395 - timing_per_token_ms/adv:0.003309535731843562 - timing_per_token_ms/gen:0.17747896882184463 - timing_per_token_ms/update_actor:0.09913914584229147 - perf/total_num_tokens:148867 - perf/time_per_step:29.983076663687825 - perf/throughput:620.6292705957158 (TaskRunner pid=2026458) Training Progress: 69%|██████▉ | 69/100 [48:20<16:58, 32.84s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". To determine the correct answer, we need to analyze the structure of the product and compare it with the options. (TaskRunner pid=2026458) (TaskRunner pid=2026458) The product contains a nitrile group (–n) attached to a cyclohexyl ring with a chlorine atom and a trifluoromethyl group (–C(F)(F)F). The structure also includes a benzene ring with a chlorine atom and a trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Looking at the options: (TaskRunner pid=2026458) (TaskRunner pid=2026458) - Option A: Contains a nitrile group and a cyclohexyl ring with a chlorine atom and a trifluoromethyl group, matching the product's structure. (TaskRunner pid=2026458) - Option B: Contains a nitrile group but does not have the correct trifluoromethyl group or the correct chlorine atom placement. (TaskRunner pid=2026458) - Option C: Contains a nitrile group but does not match the correct structure of the product, especially the trifluoromethyl group. (TaskRunner pid=2026458) - Option D: Contains a nitrile group but does not have the correct structure of the product, especially the trifluoromethyl group. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2026458) step:70 - global_seqlen/min:12837 - global_seqlen/max:19669 - global_seqlen/minmax_diff:6832 - global_seqlen/balanced_min:17821 - global_seqlen/balanced_max:17831 - global_seqlen/mean:17825.125 - actor/entropy:0.2367510199546814 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.33400535583496094 - training/rollout_probs_diff_mean:0.005260511301457882 - training/rollout_probs_diff_std:0.013495014049112797 - training/rollout_actor_probs_pearson_corr:0.9982122778892517 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.70703125 - self_distillation/correct_sample_fraction:0.55859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.70703125 - rollout_corr/rollout_is_mean:0.9999286532402039 - rollout_corr/rollout_is_ratio_fraction_high:4.527823330136016e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5092745343281422e-05 - rollout_corr/rollout_is_max:2.0651345252990723 - rollout_corr/rollout_is_min:0.28940898180007935 - rollout_corr/rollout_is_std:0.04225678741931915 - rollout_corr/rollout_is_eff_sample_size:0.9982174280795298 - rollout_corr/rollout_is_seq_mean:0.9999866485595703 - rollout_corr/rollout_is_seq_std:0.00303856679238379 - rollout_corr/rollout_is_seq_max:1.0101226568222046 - rollout_corr/rollout_is_seq_min:0.9901509284973145 - rollout_corr/rollout_is_seq_max_deviation:0.01012265682220459 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2921742992475629) - rollout_corr/training_log_ppl:np.float64(0.2528939112526132) - rollout_corr/kl:np.float64(0.0014884298633148774) - rollout_corr/k3_kl:np.float64(0.0010952503159273874) - rollout_corr/rollout_ppl:np.float64(1.2902372791431844) - rollout_corr/rollout_log_ppl:np.float64(0.2514054805797059) - rollout_corr/log_ppl_diff:np.float64(0.0014884306729072705) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028520045889308676) - rollout_corr/log_ppl_diff_max:np.float64(0.0136488676071167) - rollout_corr/log_ppl_diff_min:np.float64(-0.008753955364227295) - rollout_corr/ppl_ratio:np.float64(1.0014950232580304) - rollout_corr/chi2_token:np.float64(0.0014098831452429295) - rollout_corr/chi2_seq:np.float64(0.360071137547493) - actor/pg_loss:np.float64(0.0027717185439541936) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003434370364630013) - actor/ppo_kl:np.float64(0.0005119445677905787) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.70703125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.70703125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.55859375) - self_distillation/token_reweight_w_mean:np.float64(69071.25273821992) - self_distillation/token_reweight_w_std:np.float64(1045658.8316577639) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004556903149933) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06493389664683491) - self_distillation/empty_target_batch:np.float64(0.29296875) - actor/grad_norm:np.float64(0.3887019380927086) - perf/mfu/actor:np.float64(0.0325070599234879) - perf/max_memory_allocated_gb:np.float64(124.38179159164429) - perf/max_memory_reserved_gb:np.float64(130.84765625) - perf/cpu_memory_used_gb:np.float64(145.87537002563477) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07272307414099477) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8114142857142858) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06774926114873739) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.712047619047619) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0541181009825183) - val-aux/sciknoweval/score/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/score/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/score/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07272307414099477) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8114142857142858) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06774926114873739) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.712047619047619) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0541181009825183) - val-core/sciknoweval/acc/mean@16:np.float64(0.6851190476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.1354593314121246) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7372809523809524) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.09687814765870191) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6334190476190477) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1304254368473761) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6852190476190475) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.13554201698632168) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7717095238095236) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0626549022097885) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5802904761904761) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11571033595984151) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.697847619047619) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10316342544885265) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7940761904761904) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.040787555686648956) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5327809523809524) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09421852437209087) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7060047619047619) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07272307414099477) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8114142857142858) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02829417180722469) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4923904761904762) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06774926114873739) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.712047619047619) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0541181009825183) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:1 - critic/score/mean:0.55859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.55859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036373515613377094 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0036373515613377094 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:258.81640625 - response_length/max:938.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:258.81640625 - response_length_non_aborted/max:938.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.21875 - prompt_length/max:701.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016323477029800415 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2699729204177856) - timing_s/agent_loop/generate_sequences/max:np.float64(6.416786439716816) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.761028841654479) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.416786439716816) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:938 - timing_s/gen:12.333267584443092 - timing_s/reward:0.05591795593500137 - timing_s/old_log_prob:2.2401132248342037 - timing_s/adv:0.506872920319438 - timing_s/update_actor:15.221702938899398 - timing_s/step:30.44651348516345 - timing_s/testing:81.57863584533334 - timing_s/save_checkpoint:6.6424451023340225 - timing_s/stop_profile:0.00010693445801734924 - timing_per_token_ms/adv:0.003554483631387143 - timing_per_token_ms/gen:0.18614286165149482 - timing_per_token_ms/update_actor:0.1067433113295096 - perf/total_num_tokens:142601 - perf/time_per_step:30.44651348516345 - perf/throughput:585.4570182128132 (TaskRunner pid=2026458) Training Progress: 70%|███████ | 70/100 [50:19<29:18, 58.61s/it] (TaskRunner pid=2026458) step:71 - global_seqlen/min:15483 - global_seqlen/max:28966 - global_seqlen/minmax_diff:13483 - global_seqlen/balanced_min:18380 - global_seqlen/balanced_max:25891 - global_seqlen/mean:19333.5 - actor/entropy:0.21173985302448273 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4046512842178345 - training/rollout_probs_diff_mean:0.0045140874572098255 - training/rollout_probs_diff_std:0.012615382671356201 - training/rollout_actor_probs_pearson_corr:0.9983184337615967 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.73828125 - self_distillation/correct_sample_fraction:0.5859375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.73828125 - rollout_corr/rollout_is_mean:0.9998964667320251 - rollout_corr/rollout_is_ratio_fraction_high:1.2943308320245706e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.471654342021793e-05 - rollout_corr/rollout_is_max:2.0498809814453125 - rollout_corr/rollout_is_min:0.38866347074508667 - rollout_corr/rollout_is_std:0.03843024745583534 - rollout_corr/rollout_is_eff_sample_size:0.9985250563653204 - rollout_corr/rollout_is_seq_mean:0.9998904466629028 - rollout_corr/rollout_is_seq_std:0.0028965510427951813 - rollout_corr/rollout_is_seq_max:1.0107746124267578 - rollout_corr/rollout_is_seq_min:0.9922298192977905 - rollout_corr/rollout_is_seq_max_deviation:0.010774612426757812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2945193266496062) - rollout_corr/training_log_ppl:np.float64(0.2542406297643538) - rollout_corr/kl:np.float64(0.0010824492369110317) - rollout_corr/k3_kl:np.float64(0.0011051825894128342) - rollout_corr/rollout_ppl:np.float64(1.293056899216026) - rollout_corr/rollout_log_ppl:np.float64(0.2531581802359142) - rollout_corr/log_ppl_diff:np.float64(0.0010824495284396107) - rollout_corr/log_ppl_abs_diff:np.float64(0.00271785432141769) - rollout_corr/log_ppl_diff_max:np.float64(0.014131128787994385) - rollout_corr/log_ppl_diff_min:np.float64(-0.009531468152999878) - rollout_corr/ppl_ratio:np.float64(1.0010888520628214) - rollout_corr/chi2_token:np.float64(0.002212358172982931) - rollout_corr/chi2_seq:np.float64(0.6793141439557076) - actor/pg_loss:np.float64(0.002975177369080484) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000972424295014207) - actor/ppo_kl:np.float64(7.461081707615946e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.73828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.73828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5859375) - self_distillation/token_reweight_w_mean:np.float64(134136.12361396686) - self_distillation/token_reweight_w_std:np.float64(1624645.7259825736) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988281566184014) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07092063533491455) - self_distillation/empty_target_batch:np.float64(0.26171875) - actor/grad_norm:np.float64(0.4537533223628998) - perf/mfu/actor:np.float64(0.03374089352858298) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.65079879760742) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:1 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.06832286715507507 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.06832286715507507 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.796875 - response_length/max:8192.0 - response_length/min:93.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:301.796875 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:302.375 - prompt_length/max:666.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.677132427692413e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1272850260138512) - timing_s/agent_loop/generate_sequences/max:np.float64(47.67028555460274) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.04078216401831) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(47.67028555460274) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:666 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:53.34413158334792 - timing_s/reward:0.05393144115805626 - timing_s/old_log_prob:2.626548172906041 - timing_s/adv:0.5594039708375931 - timing_s/update_actor:16.17577089741826 - timing_s/step:72.8568678945303 - timing_s/stop_profile:2.8194859623908997e-05 - timing_per_token_ms/adv:0.0036168048389944468 - timing_per_token_ms/gen:0.6904495415913527 - timing_per_token_ms/update_actor:0.1045838240451694 - perf/total_num_tokens:154668 - perf/time_per_step:72.8568678945303 - perf/throughput:265.3627661840711 (TaskRunner pid=2026458) Training Progress: 71%|███████ | 71/100 [51:32<30:23, 62.89s/it] (TaskRunner pid=2026458) step:72 - global_seqlen/min:14180 - global_seqlen/max:19404 - global_seqlen/minmax_diff:5224 - global_seqlen/balanced_min:16784 - global_seqlen/balanced_max:16786 - global_seqlen/mean:16785.25 - actor/entropy:0.2532234191894531 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.37658625841140747 - training/rollout_probs_diff_mean:0.005358405876904726 - training/rollout_probs_diff_std:0.01324445940554142 - training/rollout_actor_probs_pearson_corr:0.9983224272727966 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83984375 - self_distillation/correct_sample_fraction:0.71484375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83984375 - rollout_corr/rollout_is_mean:1.000357747077942 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.6467605829238892 - rollout_corr/rollout_is_min:0.5746954679489136 - rollout_corr/rollout_is_std:0.04072979837656021 - rollout_corr/rollout_is_eff_sample_size:0.9983450191216581 - rollout_corr/rollout_is_seq_mean:1.0003376007080078 - rollout_corr/rollout_is_seq_std:0.0029273421969264746 - rollout_corr/rollout_is_seq_max:1.0102252960205078 - rollout_corr/rollout_is_seq_min:0.9919271469116211 - rollout_corr/rollout_is_seq_max_deviation:0.010225296020507812 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3001699862070382) - rollout_corr/training_log_ppl:np.float64(0.25781238308991306) - rollout_corr/kl:np.float64(0.0006664992815785808) - rollout_corr/k3_kl:np.float64(0.0009609586412011595) - rollout_corr/rollout_ppl:np.float64(1.2992715733125806) - rollout_corr/rollout_log_ppl:np.float64(0.25714588271512184) - rollout_corr/log_ppl_diff:np.float64(0.0006665003747912124) - rollout_corr/log_ppl_abs_diff:np.float64(0.002382488935836591) - rollout_corr/log_ppl_diff_max:np.float64(0.009035348892211914) - rollout_corr/log_ppl_diff_min:np.float64(-0.009852290153503418) - rollout_corr/ppl_ratio:np.float64(1.0006713322363794) - rollout_corr/chi2_token:np.float64(0.0025634136982262135) - rollout_corr/chi2_seq:np.float64(1.15624994575046) - actor/pg_loss:np.float64(0.003654895001091063) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009709688329166966) - actor/ppo_kl:np.float64(0.00014433233151223135) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83984375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83984375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_w_mean:np.float64(213152.86371882842) - self_distillation/token_reweight_w_std:np.float64(2646677.052918058) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9974161058198661) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09049051211331971) - self_distillation/empty_target_batch:np.float64(0.16015625) - actor/grad_norm:np.float64(0.43892450630664825) - perf/mfu/actor:np.float64(0.03149296183920366) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.40920639038086) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:1 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009050944820046425 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009050944820046425 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:237.6953125 - response_length/max:467.0 - response_length/min:111.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:237.6953125 - response_length_non_aborted/max:467.0 - response_length_non_aborted/min:111.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.84375 - prompt_length/max:620.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.147497773170471e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3375506270676851) - timing_s/agent_loop/generate_sequences/max:np.float64(4.0367792677134275) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.622172743664123) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.0367792677134275) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:326 - timing_s/agent_loop/slowest/response_length:467 - timing_s/gen:10.345969874411821 - timing_s/reward:0.05066956579685211 - timing_s/old_log_prob:2.2474361546337605 - timing_s/adv:0.4828864671289921 - timing_s/update_actor:14.702434957027435 - timing_s/step:27.925694411620498 - timing_s/stop_profile:3.0016526579856873e-05 - timing_per_token_ms/adv:0.0035960625186472655 - timing_per_token_ms/gen:0.17002415570109813 - timing_per_token_ms/update_actor:0.1094892461910564 - perf/total_num_tokens:134282 - perf/time_per_step:27.925694411620498 - perf/throughput:601.0683119491305 (TaskRunner pid=2026458) Training Progress: 72%|███████▏ | 72/100 [52:00<24:27, 52.41s/it] (TaskRunner pid=2026458) step:73 - global_seqlen/min:15559 - global_seqlen/max:20190 - global_seqlen/minmax_diff:4631 - global_seqlen/balanced_min:17173 - global_seqlen/balanced_max:17174 - global_seqlen/mean:17173.5 - actor/entropy:0.24391600489616394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135395884513855 - training/rollout_probs_diff_mean:0.0049339784309268 - training/rollout_probs_diff_std:0.012776896357536316 - training/rollout_actor_probs_pearson_corr:0.998493492603302 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:1.000167965888977 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.0463656003121287e-05 - rollout_corr/rollout_is_max:1.767087697982788 - rollout_corr/rollout_is_min:0.47380420565605164 - rollout_corr/rollout_is_std:0.03922998905181885 - rollout_corr/rollout_is_eff_sample_size:0.9984639671449654 - rollout_corr/rollout_is_seq_mean:1.0002386569976807 - rollout_corr/rollout_is_seq_std:0.0029904802795499563 - rollout_corr/rollout_is_seq_max:1.0164623260498047 - rollout_corr/rollout_is_seq_min:0.9890998005867004 - rollout_corr/rollout_is_seq_max_deviation:0.016462326049804688 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2968735643662512) - rollout_corr/training_log_ppl:np.float64(0.25511286556138657) - rollout_corr/kl:np.float64(0.0009459122673263209) - rollout_corr/k3_kl:np.float64(0.0009600993125502555) - rollout_corr/rollout_ppl:np.float64(1.2955679851584136) - rollout_corr/rollout_log_ppl:np.float64(0.2541669542551972) - rollout_corr/log_ppl_diff:np.float64(0.0009459113061893731) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024720369547139853) - rollout_corr/log_ppl_diff_max:np.float64(0.012520670890808105) - rollout_corr/log_ppl_diff_min:np.float64(-0.0084361732006073) - rollout_corr/ppl_ratio:np.float64(1.0009514002595097) - rollout_corr/chi2_token:np.float64(0.001954789971932769) - rollout_corr/chi2_seq:np.float64(0.6583140406291932) - actor/pg_loss:np.float64(0.0032611776841804385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000914203752472531) - actor/ppo_kl:np.float64(0.00038008014968227144) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.546875) - self_distillation/token_reweight_w_mean:np.float64(311191.56539016124) - self_distillation/token_reweight_w_std:np.float64(2860058.778589467) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0015116163995117) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06564340484328568) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.42411575466394424) - perf/mfu/actor:np.float64(0.031904336867600726) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.16930770874023) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:1 - critic/score/mean:0.546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.001247106003575027 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.001247106003575027 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:256.453125 - response_length/max:556.0 - response_length/min:109.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.453125 - response_length_non_aborted/max:556.0 - response_length_non_aborted/min:109.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.21875 - prompt_length/max:557.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.804398536682129e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1476398799568415) - timing_s/agent_loop/generate_sequences/max:np.float64(4.690207960084081) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7287142683780985) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.690207960084081) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:229 - timing_s/agent_loop/slowest/response_length:556 - timing_s/gen:10.320297587662935 - timing_s/reward:0.0441241730004549 - timing_s/old_log_prob:2.164985027164221 - timing_s/adv:0.4551666658371687 - timing_s/update_actor:14.973258456215262 - timing_s/step:27.992081409320235 - timing_s/stop_profile:2.880394458770752e-05 - timing_per_token_ms/adv:0.003313001614676454 - timing_per_token_ms/gen:0.15719700218824917 - timing_per_token_ms/update_actor:0.10898519853418975 - perf/total_num_tokens:137388 - perf/time_per_step:27.992081409320235 - perf/throughput:613.5127913096851 (TaskRunner pid=2026458) Training Progress: 73%|███████▎ | 73/100 [52:28<20:17, 45.09s/it] (TaskRunner pid=2026458) step:74 - global_seqlen/min:13447 - global_seqlen/max:21573 - global_seqlen/minmax_diff:8126 - global_seqlen/balanced_min:16831 - global_seqlen/balanced_max:16917 - global_seqlen/mean:16843.875 - actor/entropy:0.24996380507946014 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3424726128578186 - training/rollout_probs_diff_mean:0.004949183203279972 - training/rollout_probs_diff_std:0.012561521492898464 - training/rollout_actor_probs_pearson_corr:0.9985408782958984 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.74609375 - self_distillation/correct_sample_fraction:0.62109375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.74609375 - rollout_corr/rollout_is_mean:0.9998319149017334 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.6325607399921864e-05 - rollout_corr/rollout_is_max:1.7145717144012451 - rollout_corr/rollout_is_min:0.17540410161018372 - rollout_corr/rollout_is_std:0.03882066160440445 - rollout_corr/rollout_is_eff_sample_size:0.9984948673727597 - rollout_corr/rollout_is_seq_mean:0.9999223947525024 - rollout_corr/rollout_is_seq_std:0.0029323359485715628 - rollout_corr/rollout_is_seq_max:1.009851336479187 - rollout_corr/rollout_is_seq_min:0.9908139109611511 - rollout_corr/rollout_is_seq_max_deviation:0.009851336479187012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3159835655242205) - rollout_corr/training_log_ppl:np.float64(0.26891483918007) - rollout_corr/kl:np.float64(0.0010735450956096404) - rollout_corr/k3_kl:np.float64(0.0011378318484958072) - rollout_corr/rollout_ppl:np.float64(1.3145075496286154) - rollout_corr/rollout_log_ppl:np.float64(0.26784129165753257) - rollout_corr/log_ppl_diff:np.float64(0.0010735475225374103) - rollout_corr/log_ppl_abs_diff:np.float64(0.0030432986677624285) - rollout_corr/log_ppl_diff_max:np.float64(0.012433648109436035) - rollout_corr/log_ppl_diff_min:np.float64(-0.009890466928482056) - rollout_corr/ppl_ratio:np.float64(1.0010810112580657) - rollout_corr/chi2_token:np.float64(0.0024086034391075373) - rollout_corr/chi2_seq:np.float64(1.130532833514735) - actor/pg_loss:np.float64(0.0028035463765263557) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007331344913836801) - actor/ppo_kl:np.float64(0.00016281594710676472) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.74609375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.74609375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.62109375) - self_distillation/token_reweight_w_mean:np.float64(183593.82668150146) - self_distillation/token_reweight_w_std:np.float64(2093931.331328835) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001840919721872) - self_distillation/token_reweight_w_clip_frac:np.float64(0.060318807751173154) - self_distillation/empty_target_batch:np.float64(0.25390625) - actor/grad_norm:np.float64(0.3842148035764694) - perf/mfu/actor:np.float64(0.031244242735268143) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2487654685974) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:1 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0016619311645627022 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0016619311645627022 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:252.96484375 - response_length/max:993.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:252.96484375 - response_length_non_aborted/max:993.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.40625 - prompt_length/max:665.0 - prompt_length/min:162.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7175992727279663e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.240340730175376) - timing_s/agent_loop/generate_sequences/max:np.float64(6.539421858265996) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.619629321692628) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.539421858265996) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:291 - timing_s/agent_loop/slowest/response_length:993 - timing_s/gen:12.270534712821245 - timing_s/reward:0.04380444623529911 - timing_s/old_log_prob:2.2239825148135424 - timing_s/adv:0.5528004374355078 - timing_s/update_actor:15.180137295275927 - timing_s/step:30.30796923674643 - timing_s/stop_profile:5.2906572818756104e-05 - timing_per_token_ms/adv:0.004102384675701908 - timing_per_token_ms/gen:0.18947999062402515 - timing_per_token_ms/update_actor:0.11265324409671117 - perf/total_num_tokens:134751 - perf/time_per_step:30.30796923674643 - perf/throughput:555.7572950014052 (TaskRunner pid=2026458) Training Progress: 74%|███████▍ | 74/100 [52:58<17:37, 40.66s/it] (TaskRunner pid=2026458) step:75 - global_seqlen/min:14038 - global_seqlen/max:20039 - global_seqlen/minmax_diff:6001 - global_seqlen/balanced_min:18453 - global_seqlen/balanced_max:18459 - global_seqlen/mean:18457.625 - actor/entropy:0.24194937944412231 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4665491580963135 - training/rollout_probs_diff_mean:0.005109612364321947 - training/rollout_probs_diff_std:0.013375498354434967 - training/rollout_actor_probs_pearson_corr:0.9982732534408569 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9997668862342834 - rollout_corr/rollout_is_ratio_fraction_high:2.9664348403457552e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.449652260518633e-05 - rollout_corr/rollout_is_max:2.278925657272339 - rollout_corr/rollout_is_min:0.3029129207134247 - rollout_corr/rollout_is_std:0.040616225451231 - rollout_corr/rollout_is_eff_sample_size:0.9983523857083623 - rollout_corr/rollout_is_seq_mean:0.9997424483299255 - rollout_corr/rollout_is_seq_std:0.002785229589790106 - rollout_corr/rollout_is_seq_max:1.00969660282135 - rollout_corr/rollout_is_seq_min:0.9920895099639893 - rollout_corr/rollout_is_seq_max_deviation:0.009696602821350098 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2928350591100752) - rollout_corr/training_log_ppl:np.float64(0.2525325442838948) - rollout_corr/kl:np.float64(0.0011320576819571215) - rollout_corr/k3_kl:np.float64(0.0010110275231340893) - rollout_corr/rollout_ppl:np.float64(1.2913245973177254) - rollout_corr/rollout_log_ppl:np.float64(0.2514004855765961) - rollout_corr/log_ppl_diff:np.float64(0.001132058707298711) - rollout_corr/log_ppl_abs_diff:np.float64(0.002472092892276123) - rollout_corr/log_ppl_diff_max:np.float64(0.01102399080991745) - rollout_corr/log_ppl_diff_min:np.float64(-0.008037805557250977) - rollout_corr/ppl_ratio:np.float64(1.0011374710593373) - rollout_corr/chi2_token:np.float64(0.0017729024402797222) - rollout_corr/chi2_seq:np.float64(0.6597908593248576) - actor/pg_loss:np.float64(0.0018322557443752885) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036231086232874077) - actor/ppo_kl:np.float64(-3.7132614664869834e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(138490.71220888244) - self_distillation/token_reweight_w_std:np.float64(1204680.9713628318) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000044918153435) - self_distillation/token_reweight_w_clip_frac:np.float64(0.041466833936283365) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.29594044387340546) - perf/mfu/actor:np.float64(0.03412545800281451) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2256622314453) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:6.489075894933194e-05 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:6.489075894933194e-05 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:263.36328125 - response_length/max:680.0 - response_length/min:127.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:263.36328125 - response_length_non_aborted/max:680.0 - response_length_non_aborted/min:127.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.4375 - prompt_length/max:608.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.105005741119385e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.53872056491673) - timing_s/agent_loop/generate_sequences/max:np.float64(5.3331572357565165) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.911040708415385) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.3331572357565165) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:367 - timing_s/agent_loop/slowest/response_length:680 - timing_s/gen:11.2033476177603 - timing_s/reward:0.055844465270638466 - timing_s/old_log_prob:2.219141924753785 - timing_s/adv:0.4940295107662678 - timing_s/update_actor:15.452317954972386 - timing_s/step:29.51427774503827 - timing_s/stop_profile:0.0001329611986875534 - timing_per_token_ms/adv:0.003345700697992481 - timing_per_token_ms/gen:0.16617000070838908 - timing_per_token_ms/update_actor:0.10464725252417623 - perf/total_num_tokens:147661 - perf/time_per_step:29.51427774503827 - perf/throughput:625.3795251046915 (TaskRunner pid=2026458) Training Progress: 75%|███████▌ | 75/100 [53:28<15:33, 37.33s/it] (TaskRunner pid=2026458) step:76 - global_seqlen/min:15400 - global_seqlen/max:20933 - global_seqlen/minmax_diff:5533 - global_seqlen/balanced_min:18318 - global_seqlen/balanced_max:18443 - global_seqlen/mean:18349.25 - actor/entropy:0.23565568029880524 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6839720010757446 - training/rollout_probs_diff_mean:0.00476441252976656 - training/rollout_probs_diff_std:0.01295106578618288 - training/rollout_actor_probs_pearson_corr:0.9984387159347534 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.65625 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.65625 - rollout_corr/rollout_is_mean:0.999829113483429 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.072877345606685e-05 - rollout_corr/rollout_is_max:1.8591279983520508 - rollout_corr/rollout_is_min:0.14236801862716675 - rollout_corr/rollout_is_std:0.038981545716524124 - rollout_corr/rollout_is_eff_sample_size:0.9984822693566601 - rollout_corr/rollout_is_seq_mean:0.9998883008956909 - rollout_corr/rollout_is_seq_std:0.0026900693774223328 - rollout_corr/rollout_is_seq_max:1.0127232074737549 - rollout_corr/rollout_is_seq_min:0.9910237193107605 - rollout_corr/rollout_is_seq_max_deviation:0.012723207473754883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3050027494318783) - rollout_corr/training_log_ppl:np.float64(0.26096004332066514) - rollout_corr/kl:np.float64(0.001120272666678801) - rollout_corr/k3_kl:np.float64(0.000998899616035942) - rollout_corr/rollout_ppl:np.float64(1.3034690767526627) - rollout_corr/rollout_log_ppl:np.float64(0.25983976959832944) - rollout_corr/log_ppl_diff:np.float64(0.0011202737223356962) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024523677420802414) - rollout_corr/log_ppl_diff_max:np.float64(0.01079338788986206) - rollout_corr/log_ppl_diff_min:np.float64(-0.011696979403495789) - rollout_corr/ppl_ratio:np.float64(1.001125503797084) - rollout_corr/chi2_token:np.float64(0.0016837881412357092) - rollout_corr/chi2_seq:np.float64(0.4817654227372259) - actor/pg_loss:np.float64(0.0022594364127144217) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005095393812553084) - actor/ppo_kl:np.float64(0.000131376795138749) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.65625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.65625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(86914.96509032976) - self_distillation/token_reweight_w_std:np.float64(1369863.7220702441) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9999393313191831) - self_distillation/token_reweight_w_clip_frac:np.float64(0.041579058073693886) - self_distillation/empty_target_batch:np.float64(0.34375) - actor/grad_norm:np.float64(0.3407774642109871) - perf/mfu/actor:np.float64(0.034170829781458016) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.09953689575195) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002358874771744013 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002358874771744013 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:287.7265625 - response_length/max:956.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:287.7265625 - response_length_non_aborted/max:956.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:285.6875 - prompt_length/max:526.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015220046043395996 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3281499948352575) - timing_s/agent_loop/generate_sequences/max:np.float64(6.804745152592659) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0793025807652157) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.804745152592659) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:306 - timing_s/agent_loop/slowest/response_length:956 - timing_s/gen:12.672567792236805 - timing_s/reward:0.05679433047771454 - timing_s/old_log_prob:2.200064066797495 - timing_s/adv:0.5124469976872206 - timing_s/update_actor:15.322908757254481 - timing_s/step:30.852899638935924 - timing_s/stop_profile:0.00013393722474575043 - timing_per_token_ms/adv:0.0034909260438929423 - timing_per_token_ms/gen:0.17204604784594754 - timing_per_token_ms/update_actor:0.10438375381319728 - perf/total_num_tokens:146794 - perf/time_per_step:30.852899638935924 - perf/throughput:594.7334031723717 (TaskRunner pid=2026458) Training Progress: 76%|███████▌ | 76/100 [53:59<14:09, 35.40s/it] (TaskRunner pid=2026458) step:77 - global_seqlen/min:16068 - global_seqlen/max:21136 - global_seqlen/minmax_diff:5068 - global_seqlen/balanced_min:18347 - global_seqlen/balanced_max:18355 - global_seqlen/mean:18353.125 - actor/entropy:0.24358664453029633 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3763711750507355 - training/rollout_probs_diff_mean:0.005085653159767389 - training/rollout_probs_diff_std:0.012961719185113907 - training/rollout_actor_probs_pearson_corr:0.9984139204025269 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.8125 - self_distillation/correct_sample_fraction:0.7265625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.8125 - rollout_corr/rollout_is_mean:0.9999815821647644 - rollout_corr/rollout_is_ratio_fraction_high:1.5232524674502201e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.61626215535216e-05 - rollout_corr/rollout_is_max:2.443542003631592 - rollout_corr/rollout_is_min:0.0030977046117186546 - rollout_corr/rollout_is_std:0.04062869772315025 - rollout_corr/rollout_is_eff_sample_size:0.9983521480748362 - rollout_corr/rollout_is_seq_mean:1.0000320672988892 - rollout_corr/rollout_is_seq_std:0.0030800357926636934 - rollout_corr/rollout_is_seq_max:1.008837342262268 - rollout_corr/rollout_is_seq_min:0.9924508333206177 - rollout_corr/rollout_is_seq_max_deviation:0.008837342262268066 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.30547040887177) - rollout_corr/training_log_ppl:np.float64(0.262247261358425) - rollout_corr/kl:np.float64(0.001232846368517393) - rollout_corr/k3_kl:np.float64(0.0012621798805412254) - rollout_corr/rollout_ppl:np.float64(1.303758448921144) - rollout_corr/rollout_log_ppl:np.float64(0.2610144141508499) - rollout_corr/log_ppl_diff:np.float64(0.0012328472075751051) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028479194588726386) - rollout_corr/log_ppl_diff_max:np.float64(0.01525259017944336) - rollout_corr/log_ppl_diff_min:np.float64(-0.007772982120513916) - rollout_corr/ppl_ratio:np.float64(1.0012400422710925) - rollout_corr/chi2_token:np.float64(0.0023893536999821663) - rollout_corr/chi2_seq:np.float64(0.8159184374380857) - actor/pg_loss:np.float64(0.002790838829241693) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009325358823843999) - actor/ppo_kl:np.float64(0.0003359223030457992) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.8125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.8125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.7265625) - self_distillation/token_reweight_w_mean:np.float64(180959.5167653456) - self_distillation/token_reweight_w_std:np.float64(2185127.907001757) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988750158809125) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06124208416440524) - self_distillation/empty_target_batch:np.float64(0.1875) - actor/grad_norm:np.float64(0.4322587177157402) - perf/mfu/actor:np.float64(0.034241717008689325) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1767692565918) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0021534981206059456 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.0021534981206059456 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:256.44140625 - response_length/max:782.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:256.44140625 - response_length_non_aborted/max:782.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:317.09375 - prompt_length/max:716.0 - prompt_length/min:173.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013564899563789368 - timing_s/agent_loop/generate_sequences/min:np.float64(1.442766698077321) - timing_s/agent_loop/generate_sequences/max:np.float64(5.666013702750206) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7960805580951273) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.666013702750206) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:368 - timing_s/agent_loop/slowest/response_length:782 - timing_s/gen:11.443953750655055 - timing_s/reward:0.054556334391236305 - timing_s/old_log_prob:2.218324951827526 - timing_s/adv:0.4792825896292925 - timing_s/update_actor:15.45035495236516 - timing_s/step:29.73319257237017 - timing_s/stop_profile:2.9521062970161438e-05 - timing_per_token_ms/adv:0.0032643118653450875 - timing_per_token_ms/gen:0.1743203057267446 - timing_per_token_ms/update_actor:0.10522972894510581 - perf/total_num_tokens:146825 - perf/time_per_step:29.73319257237017 - perf/throughput:617.2604894455499 (TaskRunner pid=2026458) Training Progress: 77%|███████▋ | 77/100 [54:28<12:55, 33.71s/it] (TaskRunner pid=2026458) step:78 - global_seqlen/min:15356 - global_seqlen/max:19696 - global_seqlen/minmax_diff:4340 - global_seqlen/balanced_min:17649 - global_seqlen/balanced_max:17655 - global_seqlen/mean:17652.75 - actor/entropy:0.23699037730693817 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.39975830912590027 - training/rollout_probs_diff_mean:0.005081570707261562 - training/rollout_probs_diff_std:0.013243689201772213 - training/rollout_actor_probs_pearson_corr:0.9983371496200562 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.65234375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:0.9999217987060547 - rollout_corr/rollout_is_ratio_fraction_high:4.307003109715879e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.307003109715879e-05 - rollout_corr/rollout_is_max:2.9357402324676514 - rollout_corr/rollout_is_min:0.4458056390285492 - rollout_corr/rollout_is_std:0.04097927734255791 - rollout_corr/rollout_is_eff_sample_size:0.9983230388235971 - rollout_corr/rollout_is_seq_mean:1.0000085830688477 - rollout_corr/rollout_is_seq_std:0.002981689525768161 - rollout_corr/rollout_is_seq_max:1.0086852312088013 - rollout_corr/rollout_is_seq_min:0.9888802170753479 - rollout_corr/rollout_is_seq_max_deviation:0.0111197829246521 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3004110623151064) - rollout_corr/training_log_ppl:np.float64(0.257169630160206) - rollout_corr/kl:np.float64(0.0012380461494814199) - rollout_corr/k3_kl:np.float64(0.001083872083654569) - rollout_corr/rollout_ppl:np.float64(1.2987679266370833) - rollout_corr/rollout_log_ppl:np.float64(0.25593158273841254) - rollout_corr/log_ppl_diff:np.float64(0.0012380474217934534) - rollout_corr/log_ppl_abs_diff:np.float64(0.002574279482359998) - rollout_corr/log_ppl_diff_max:np.float64(0.015157654881477356) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076891034841537476) - rollout_corr/ppl_ratio:np.float64(1.0012440220452845) - rollout_corr/chi2_token:np.float64(0.001788182882592082) - rollout_corr/chi2_seq:np.float64(1.9581220541149378) - actor/pg_loss:np.float64(0.0033810948953032494) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009038109546963824) - actor/ppo_kl:np.float64(0.00033419562218028886) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.65234375) - self_distillation/token_reweight_w_mean:np.float64(189436.75045557343) - self_distillation/token_reweight_w_std:np.float64(2503367.794994272) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009928282815963) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07160566429956816) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.3780764937400818) - perf/mfu/actor:np.float64(0.03291894268049457) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.0838165283203) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0019830160308629274 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0019830160308629274 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:272.0859375 - response_length/max:624.0 - response_length/min:113.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:272.0859375 - response_length_non_aborted/max:624.0 - response_length_non_aborted/min:113.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5625 - prompt_length/max:449.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.438022971153259e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3432003632187843) - timing_s/agent_loop/generate_sequences/max:np.float64(5.082683224231005) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9419547133074957) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.082683224231005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:190 - timing_s/agent_loop/slowest/response_length:624 - timing_s/gen:10.891948658972979 - timing_s/reward:0.054630380123853683 - timing_s/old_log_prob:2.192325107753277 - timing_s/adv:0.4803696293383837 - timing_s/update_actor:15.322288809344172 - timing_s/step:29.02985128760338 - timing_s/stop_profile:0.00014253705739974976 - timing_per_token_ms/adv:0.003401521217220997 - timing_per_token_ms/gen:0.15637219196274413 - timing_per_token_ms/update_actor:0.10849788849714756 - perf/total_num_tokens:141222 - perf/time_per_step:29.02985128760338 - perf/throughput:608.0895773495835 (TaskRunner pid=2026458) Training Progress: 78%|███████▊ | 78/100 [54:57<11:51, 32.32s/it] (TaskRunner pid=2026458) step:79 - global_seqlen/min:13318 - global_seqlen/max:19723 - global_seqlen/minmax_diff:6405 - global_seqlen/balanced_min:16851 - global_seqlen/balanced_max:16855 - global_seqlen/mean:16852.75 - actor/entropy:0.2380763292312622 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24131831526756287 - training/rollout_probs_diff_mean:0.00509747164323926 - training/rollout_probs_diff_std:0.01309633906930685 - training/rollout_actor_probs_pearson_corr:0.998324453830719 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.78125 - self_distillation/correct_sample_fraction:0.59765625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.78125 - rollout_corr/rollout_is_mean:0.999850869178772 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.4865467164781876e-05 - rollout_corr/rollout_is_max:1.6486144065856934 - rollout_corr/rollout_is_min:0.48596420884132385 - rollout_corr/rollout_is_std:0.04019506648182869 - rollout_corr/rollout_is_eff_sample_size:0.998386606117186 - rollout_corr/rollout_is_seq_mean:0.9999116659164429 - rollout_corr/rollout_is_seq_std:0.0028780419379472733 - rollout_corr/rollout_is_seq_max:1.0085704326629639 - rollout_corr/rollout_is_seq_min:0.9908472299575806 - rollout_corr/rollout_is_seq_max_deviation:0.009152770042419434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2909451411105692) - rollout_corr/training_log_ppl:np.float64(0.250693125941325) - rollout_corr/kl:np.float64(0.0009612457183649781) - rollout_corr/k3_kl:np.float64(0.0010165603632685816) - rollout_corr/rollout_ppl:np.float64(1.2896752119995654) - rollout_corr/rollout_log_ppl:np.float64(0.24973187799332663) - rollout_corr/log_ppl_diff:np.float64(0.0009612479479983449) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024955651024356484) - rollout_corr/log_ppl_diff_max:np.float64(0.008482903242111206) - rollout_corr/log_ppl_diff_min:np.float64(-0.007829368114471436) - rollout_corr/ppl_ratio:np.float64(1.0009662464726716) - rollout_corr/chi2_token:np.float64(0.002142100827768445) - rollout_corr/chi2_seq:np.float64(0.36312326532788575) - actor/pg_loss:np.float64(0.0036202146438881755) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008493167902088317) - actor/ppo_kl:np.float64(1.813888876256442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.78125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.78125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.59765625) - self_distillation/token_reweight_w_mean:np.float64(207000.82824485935) - self_distillation/token_reweight_w_std:np.float64(2774328.0958690653) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004315504338592) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07667552542989142) - self_distillation/empty_target_batch:np.float64(0.21875) - actor/grad_norm:np.float64(0.5015818998217583) - perf/mfu/actor:np.float64(0.031159245026246143) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.08302688598633) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:1 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000724691548384726 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000724691548384726 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:262.7734375 - response_length/max:630.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:262.7734375 - response_length_non_aborted/max:630.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:766.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013626180589199066 - timing_s/agent_loop/generate_sequences/min:np.float64(1.074292877689004) - timing_s/agent_loop/generate_sequences/max:np.float64(4.940646631643176) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.7851551758358255) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(4.940646631643176) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:276 - timing_s/agent_loop/slowest/response_length:630 - timing_s/gen:11.244631016626954 - timing_s/reward:0.05328981205821037 - timing_s/old_log_prob:2.2739136423915625 - timing_s/adv:0.5032551754266024 - timing_s/update_actor:15.453622449189425 - timing_s/step:29.61371649429202 - timing_s/stop_profile:0.00012503564357757568 - timing_per_token_ms/adv:0.0037327377981828066 - timing_per_token_ms/gen:0.16715669714028472 - timing_per_token_ms/update_actor:0.1146224091705317 - perf/total_num_tokens:134822 - perf/time_per_step:29.61371649429202 - perf/throughput:569.0859505340484 (TaskRunner pid=2026458) Training Progress: 79%|███████▉ | 79/100 [55:27<11:02, 31.52s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a nitrile group (n1) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group (C(F)(F)F). (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains a nitrile group (n1) and a substituted benzene ring with a chlorine atom and a trifluoromethyl group, matching the product structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B: "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" contains a chlorine atom but does not have the correct nitrile group and trifluoromethyl group structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C: "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" has a different structure with an oxygen atom and a different substitution pattern, not matching the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D: "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" contains a different structure with a double bond and a different substitution pattern, not matching the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it matches the structure of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2026458) step:80 - global_seqlen/min:16122 - global_seqlen/max:23121 - global_seqlen/minmax_diff:6999 - global_seqlen/balanced_min:18851 - global_seqlen/balanced_max:18860 - global_seqlen/mean:18857.375 - actor/entropy:0.22489960491657257 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3807961344718933 - training/rollout_probs_diff_mean:0.005050069652497768 - training/rollout_probs_diff_std:0.013410150073468685 - training/rollout_actor_probs_pearson_corr:0.998170018196106 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.890625 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.890625 - rollout_corr/rollout_is_mean:1.0001280307769775 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.252001963322982e-05 - rollout_corr/rollout_is_max:1.815222144126892 - rollout_corr/rollout_is_min:0.23840588331222534 - rollout_corr/rollout_is_std:0.040402136743068695 - rollout_corr/rollout_is_eff_sample_size:0.9983706838295904 - rollout_corr/rollout_is_seq_mean:1.0001842975616455 - rollout_corr/rollout_is_seq_std:0.002584717469289899 - rollout_corr/rollout_is_seq_max:1.0079708099365234 - rollout_corr/rollout_is_seq_min:0.9935450553894043 - rollout_corr/rollout_is_seq_max_deviation:0.007970809936523438 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.284651028458029) - rollout_corr/training_log_ppl:np.float64(0.2446684459282551) - rollout_corr/kl:np.float64(0.0011069629296169836) - rollout_corr/k3_kl:np.float64(0.0011388705046897485) - rollout_corr/rollout_ppl:np.float64(1.2831353461369872) - rollout_corr/rollout_log_ppl:np.float64(0.24356148090737406) - rollout_corr/log_ppl_diff:np.float64(0.0011069650208810344) - rollout_corr/log_ppl_abs_diff:np.float64(0.002547982890973799) - rollout_corr/log_ppl_diff_max:np.float64(0.011913180351257324) - rollout_corr/log_ppl_diff_min:np.float64(-0.00954301655292511) - rollout_corr/ppl_ratio:np.float64(1.0011127449106425) - rollout_corr/chi2_token:np.float64(0.0022547231055796146) - rollout_corr/chi2_seq:np.float64(0.705732943257317) - actor/pg_loss:np.float64(0.003671735990792513) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0012357970999801182) - actor/ppo_kl:np.float64(0.00041334695177397407) - actor/pg_clipfrac_lower:np.float64(2.100134406646248e-05) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.890625) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.890625) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(147860.0034218824) - self_distillation/token_reweight_w_std:np.float64(2243142.9639211292) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0017643314786255) - self_distillation/token_reweight_w_clip_frac:np.float64(0.08743216533912346) - self_distillation/empty_target_batch:np.float64(0.109375) - actor/grad_norm:np.float64(0.46084368228912354) - perf/mfu/actor:np.float64(0.0348035546181388) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.0935287475586) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/reward/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07152449490492849) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8101809523809524) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05480896953178682) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6956380952380953) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.04910459662444814) - val-aux/sciknoweval/score/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/score/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/score/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07152449490492849) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8101809523809524) - val-aux/sciknoweval/score/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05480896953178682) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6956380952380953) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.04910459662444814) - val-core/sciknoweval/acc/mean@16:np.float64(0.6788690476190476) - val-aux/sciknoweval/acc/std@16:np.float64(0.13354484169727368) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7320952380952381) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.10085829139940647) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6251047619047618) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.12484909422849347) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6789857142857142) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.13328046089893666) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7699333333333332) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06599987955078114) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5759809523809525) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10617677121952601) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6900714285714286) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1031404305918038) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7950666666666667) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04028583090873896) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5310380952380952) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08297067195315495) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6945809523809524) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07152449490492849) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8101809523809524) - val-core/sciknoweval/acc/best@16/std:np.float64(0.021522036689052344) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.49653333333333327) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05480896953178682) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6956380952380953) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.04910459662444814) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0028665580321103334 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0028665580321103334 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:275.60546875 - response_length/max:780.0 - response_length/min:128.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:275.60546875 - response_length_non_aborted/max:780.0 - response_length_non_aborted/min:128.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:313.6875 - prompt_length/max:950.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012242421507835388 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5708349235355854) - timing_s/agent_loop/generate_sequences/max:np.float64(5.78690424002707) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9946807451196946) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.78690424002707) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:635 - timing_s/agent_loop/slowest/response_length:780 - timing_s/gen:11.676643127575517 - timing_s/reward:0.060109492391347885 - timing_s/old_log_prob:2.263966953381896 - timing_s/adv:0.5860696248710155 - timing_s/update_actor:15.427983490750194 - timing_s/step:30.10305192694068 - timing_s/testing:71.47364217042923 - timing_s/save_checkpoint:6.949206165969372 - timing_s/stop_profile:0.00013513118028640747 - timing_per_token_ms/adv:0.0038848834002016158 - timing_per_token_ms/gen:0.16549703249345216 - timing_per_token_ms/update_actor:0.10226757098184526 - perf/total_num_tokens:150859 - perf/time_per_step:30.10305192694068 - perf/throughput:626.4273484883312 (TaskRunner pid=2026458) Training Progress: 80%|████████ | 80/100 [57:16<18:12, 54.65s/it] (TaskRunner pid=2026458) step:81 - global_seqlen/min:15749 - global_seqlen/max:23070 - global_seqlen/minmax_diff:7321 - global_seqlen/balanced_min:19345 - global_seqlen/balanced_max:19350 - global_seqlen/mean:19349.125 - actor/entropy:0.24268725514411926 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.0049947695806622505 - training/rollout_probs_diff_std:0.013053881004452705 - training/rollout_actor_probs_pearson_corr:0.9983764290809631 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:0.9997670650482178 - rollout_corr/rollout_is_ratio_fraction_high:1.3156336535757873e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.262534614303149e-05 - rollout_corr/rollout_is_max:2.0260822772979736 - rollout_corr/rollout_is_min:0.2609400451183319 - rollout_corr/rollout_is_std:0.03953344747424126 - rollout_corr/rollout_is_eff_sample_size:0.9984388918274393 - rollout_corr/rollout_is_seq_mean:0.999757707118988 - rollout_corr/rollout_is_seq_std:0.002598333638161421 - rollout_corr/rollout_is_seq_max:1.0065761804580688 - rollout_corr/rollout_is_seq_min:0.9917704463005066 - rollout_corr/rollout_is_seq_max_deviation:0.008229553699493408 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2969164904206991) - rollout_corr/training_log_ppl:np.float64(0.2555096454452723) - rollout_corr/kl:np.float64(0.0012044042446035519) - rollout_corr/k3_kl:np.float64(0.001068380127094315) - rollout_corr/rollout_ppl:np.float64(1.295284100342542) - rollout_corr/rollout_log_ppl:np.float64(0.25430524366674945) - rollout_corr/log_ppl_diff:np.float64(0.0012044017785228789) - rollout_corr/log_ppl_abs_diff:np.float64(0.002569538773968816) - rollout_corr/log_ppl_diff_max:np.float64(0.00988227128982544) - rollout_corr/log_ppl_diff_min:np.float64(-0.007966101169586182) - rollout_corr/ppl_ratio:np.float64(1.0012098925653845) - rollout_corr/chi2_token:np.float64(0.0018374514766037464) - rollout_corr/chi2_seq:np.float64(0.6602487270720303) - actor/pg_loss:np.float64(0.0036679430631920695) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007375691257038852) - actor/ppo_kl:np.float64(0.00012186931077984298) - actor/pg_clipfrac_lower:np.float64(8.037551197048742e-06) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(152346.46296780813) - self_distillation/token_reweight_w_std:np.float64(2223816.070475471) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0006990339607) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07411914720432833) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.43313899636268616) - perf/mfu/actor:np.float64(0.035338759221170066) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.7124252319336) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0049204700626432896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0049204700626432896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:296.91015625 - response_length/max:761.0 - response_length/min:146.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.91015625 - response_length_non_aborted/max:761.0 - response_length_non_aborted/min:146.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:307.75 - prompt_length/max:735.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.6648626923561096e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7210864126682281) - timing_s/agent_loop/generate_sequences/max:np.float64(5.891790520399809) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.159814590260794) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.891790520399809) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:423 - timing_s/agent_loop/slowest/response_length:761 - timing_s/gen:11.6525589954108 - timing_s/reward:0.05602879822254181 - timing_s/old_log_prob:2.4974898006767035 - timing_s/adv:0.5450900681316853 - timing_s/update_actor:15.660592023283243 - timing_s/step:30.51396298967302 - timing_s/stop_profile:0.00014164485037326813 - timing_per_token_ms/adv:0.0035214129071190897 - timing_per_token_ms/gen:0.15330499013815205 - timing_per_token_ms/update_actor:0.10117119006210386 - perf/total_num_tokens:154793 - perf/time_per_step:30.51396298967302 - perf/throughput:634.1072448225888 (TaskRunner pid=2026458) Training Progress: 81%|████████ | 81/100 [57:46<15:01, 47.43s/it] (TaskRunner pid=2026458) step:82 - global_seqlen/min:15004 - global_seqlen/max:21121 - global_seqlen/minmax_diff:6117 - global_seqlen/balanced_min:17893 - global_seqlen/balanced_max:17896 - global_seqlen/mean:17895.25 - actor/entropy:0.23795437812805176 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31782522797584534 - training/rollout_probs_diff_mean:0.004893430043011904 - training/rollout_probs_diff_std:0.012663507834076881 - training/rollout_actor_probs_pearson_corr:0.9984432458877563 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80078125 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80078125 - rollout_corr/rollout_is_mean:1.000130534172058 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.6648900529835373e-05 - rollout_corr/rollout_is_max:1.7479747533798218 - rollout_corr/rollout_is_min:0.3629961311817169 - rollout_corr/rollout_is_std:0.038970839232206345 - rollout_corr/rollout_is_eff_sample_size:0.998484170923625 - rollout_corr/rollout_is_seq_mean:1.0002964735031128 - rollout_corr/rollout_is_seq_std:0.002696148119866848 - rollout_corr/rollout_is_seq_max:1.0123590230941772 - rollout_corr/rollout_is_seq_min:0.9923802018165588 - rollout_corr/rollout_is_seq_max_deviation:0.012359023094177246 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.296073100529611) - rollout_corr/training_log_ppl:np.float64(0.2552825820748694) - rollout_corr/kl:np.float64(0.0009574908228102696) - rollout_corr/k3_kl:np.float64(0.0011017269548574404) - rollout_corr/rollout_ppl:np.float64(1.2947798878885806) - rollout_corr/rollout_log_ppl:np.float64(0.254325088521) - rollout_corr/log_ppl_diff:np.float64(0.0009574935538694263) - rollout_corr/log_ppl_abs_diff:np.float64(0.002439796517137438) - rollout_corr/log_ppl_diff_max:np.float64(0.014934465289115906) - rollout_corr/log_ppl_diff_min:np.float64(-0.010826826095581055) - rollout_corr/ppl_ratio:np.float64(1.0009630019776523) - rollout_corr/chi2_token:np.float64(0.002452905522659421) - rollout_corr/chi2_seq:np.float64(0.423899469897151) - actor/pg_loss:np.float64(0.0041424884693697095) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013430351091301418) - actor/ppo_kl:np.float64(0.000410171516927349) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80078125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80078125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(233069.21662305342) - self_distillation/token_reweight_w_std:np.float64(2612623.9466522) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009098192676902) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0835862404492218) - self_distillation/empty_target_batch:np.float64(0.19921875) - actor/grad_norm:np.float64(0.45514093339443207) - perf/mfu/actor:np.float64(0.03317730103506178) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.53085327148438) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.003221185877919197 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.003221185877919197 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.1640625 - response_length/max:790.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.1640625 - response_length_non_aborted/max:790.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.0625 - prompt_length/max:408.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010550394654273987 - timing_s/agent_loop/generate_sequences/min:np.float64(1.162438377737999) - timing_s/agent_loop/generate_sequences/max:np.float64(6.011627459898591) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.059660606115358) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.011627459898591) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:298 - timing_s/agent_loop/slowest/response_length:790 - timing_s/gen:12.346360467374325 - timing_s/reward:0.05675779469311237 - timing_s/old_log_prob:2.2316900305449963 - timing_s/adv:0.5058559346944094 - timing_s/update_actor:15.28704277612269 - timing_s/step:30.49560439772904 - timing_s/stop_profile:0.00011858716607093811 - timing_per_token_ms/adv:0.0035334511580895024 - timing_per_token_ms/gen:0.16450846725348867 - timing_per_token_ms/update_actor:0.10678142786579323 - perf/total_num_tokens:143162 - perf/time_per_step:30.49560439772904 - perf/throughput:586.8140787310525 (TaskRunner pid=2026458) Training Progress: 82%|████████▏ | 82/100 [58:17<12:42, 42.36s/it] (TaskRunner pid=2026458) step:83 - global_seqlen/min:15510 - global_seqlen/max:20545 - global_seqlen/minmax_diff:5035 - global_seqlen/balanced_min:18663 - global_seqlen/balanced_max:18665 - global_seqlen/mean:18664.0 - actor/entropy:0.253889262676239 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553972005844116 - training/rollout_probs_diff_mean:0.005415121093392372 - training/rollout_probs_diff_std:0.013402428478002548 - training/rollout_actor_probs_pearson_corr:0.9983406662940979 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6875 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6875 - rollout_corr/rollout_is_mean:1.0002059936523438 - rollout_corr/rollout_is_ratio_fraction_high:2.7475547540234402e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3737773770117201e-05 - rollout_corr/rollout_is_max:2.8205859661102295 - rollout_corr/rollout_is_min:0.3914983570575714 - rollout_corr/rollout_is_std:0.04187990352511406 - rollout_corr/rollout_is_eff_sample_size:0.9982497385555893 - rollout_corr/rollout_is_seq_mean:1.000227689743042 - rollout_corr/rollout_is_seq_std:0.002809274010360241 - rollout_corr/rollout_is_seq_max:1.009268045425415 - rollout_corr/rollout_is_seq_min:0.9934084415435791 - rollout_corr/rollout_is_seq_max_deviation:0.009268045425415039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3143565426580608) - rollout_corr/training_log_ppl:np.float64(0.2695897765806876) - rollout_corr/kl:np.float64(0.000838180147326284) - rollout_corr/k3_kl:np.float64(0.001138548173230447) - rollout_corr/rollout_ppl:np.float64(1.313231692649424) - rollout_corr/rollout_log_ppl:np.float64(0.26875159548944794) - rollout_corr/log_ppl_diff:np.float64(0.0008381810912396759) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024882252619136125) - rollout_corr/log_ppl_diff_max:np.float64(0.013383328914642334) - rollout_corr/log_ppl_diff_min:np.float64(-0.00783604383468628) - rollout_corr/ppl_ratio:np.float64(1.0008434464689344) - rollout_corr/chi2_token:np.float64(0.002809858415275812) - rollout_corr/chi2_seq:np.float64(0.9911151258274913) - actor/pg_loss:np.float64(0.004593900288455188) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000947352435105131) - actor/ppo_kl:np.float64(0.00014004049228333315) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(142169.04989654478) - self_distillation/token_reweight_w_std:np.float64(1783576.419247636) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9992821563500911) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07421927264658734) - self_distillation/empty_target_batch:np.float64(0.3125) - actor/grad_norm:np.float64(0.4911562651395798) - perf/mfu/actor:np.float64(0.03439597980816678) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.20166397094727) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0015317617217078805 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0015317617217078805 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:284.34375 - response_length/max:727.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:284.34375 - response_length_non_aborted/max:727.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.90625 - prompt_length/max:563.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.794160723686218e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2851199880242348) - timing_s/agent_loop/generate_sequences/max:np.float64(5.475394682958722) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0284413718109136) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.475394682958722) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:226 - timing_s/agent_loop/slowest/response_length:727 - timing_s/gen:11.69941053725779 - timing_s/reward:0.055808091536164284 - timing_s/old_log_prob:2.32892463542521 - timing_s/adv:0.5197819676250219 - timing_s/update_actor:15.535809490829706 - timing_s/step:30.227494234219193 - timing_s/stop_profile:0.00012695416808128357 - timing_per_token_ms/adv:0.0034811801303647524 - timing_per_token_ms/gen:0.16072385066020703 - timing_per_token_ms/update_actor:0.10404930274076904 - perf/total_num_tokens:149312 - perf/time_per_step:30.227494234219193 - perf/throughput:617.4511143855027 (TaskRunner pid=2026458) Training Progress: 83%|████████▎ | 83/100 [58:47<10:58, 38.73s/it] (TaskRunner pid=2026458) step:84 - global_seqlen/min:14717 - global_seqlen/max:21368 - global_seqlen/minmax_diff:6651 - global_seqlen/balanced_min:17413 - global_seqlen/balanced_max:17422 - global_seqlen/mean:17419.375 - actor/entropy:0.22988727688789368 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35856932401657104 - training/rollout_probs_diff_mean:0.005215250421315432 - training/rollout_probs_diff_std:0.013718393631279469 - training/rollout_actor_probs_pearson_corr:0.9981431365013123 - self_distillation/success_group_fraction:0.90625 - self_distillation/success_sample_fraction:0.89453125 - self_distillation/correct_sample_fraction:0.68359375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.89453125 - rollout_corr/rollout_is_mean:0.9996470212936401 - rollout_corr/rollout_is_ratio_fraction_high:1.3454785403155256e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.072871423792094e-05 - rollout_corr/rollout_is_max:2.1147913932800293 - rollout_corr/rollout_is_min:0.390633225440979 - rollout_corr/rollout_is_std:0.041462238878011703 - rollout_corr/rollout_is_eff_sample_size:0.9982825263238067 - rollout_corr/rollout_is_seq_mean:0.9997439980506897 - rollout_corr/rollout_is_seq_std:0.0025515363086014986 - rollout_corr/rollout_is_seq_max:1.0085729360580444 - rollout_corr/rollout_is_seq_min:0.9917205572128296 - rollout_corr/rollout_is_seq_max_deviation:0.008572936058044434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2788517312146723) - rollout_corr/training_log_ppl:np.float64(0.24251381697831675) - rollout_corr/kl:np.float64(0.0013615037188472456) - rollout_corr/k3_kl:np.float64(0.0013387326044949077) - rollout_corr/rollout_ppl:np.float64(1.2770492630079389) - rollout_corr/rollout_log_ppl:np.float64(0.2411523106857203) - rollout_corr/log_ppl_diff:np.float64(0.0013615062925964594) - rollout_corr/log_ppl_abs_diff:np.float64(0.002768128179013729) - rollout_corr/log_ppl_diff_max:np.float64(0.011708617210388184) - rollout_corr/log_ppl_diff_min:np.float64(-0.010206863284111023) - rollout_corr/ppl_ratio:np.float64(1.0013677747920156) - rollout_corr/chi2_token:np.float64(0.0024920080322772264) - rollout_corr/chi2_seq:np.float64(0.9803334202151746) - actor/pg_loss:np.float64(0.0052698974031955) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0015525902631452482) - actor/ppo_kl:np.float64(0.00019933821041862032) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.89453125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.89453125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.68359375) - self_distillation/token_reweight_w_mean:np.float64(172811.0304318224) - self_distillation/token_reweight_w_std:np.float64(2611723.788451826) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009630725253373) - self_distillation/token_reweight_w_clip_frac:np.float64(0.09219890262465924) - self_distillation/empty_target_batch:np.float64(0.10546875) - actor/grad_norm:np.float64(0.5317341759800911) - perf/mfu/actor:np.float64(0.03233182941898891) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.27649307250977) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011396203190088272 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011396203190088272 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:290.32421875 - response_length/max:900.0 - response_length/min:117.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:290.32421875 - response_length_non_aborted/max:900.0 - response_length_non_aborted/min:117.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.03125 - prompt_length/max:466.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003882218152284622 - timing_s/agent_loop/generate_sequences/min:np.float64(1.449787463992834) - timing_s/agent_loop/generate_sequences/max:np.float64(6.467833820730448) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1213262150922674) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.467833820730448) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:245 - timing_s/agent_loop/slowest/response_length:900 - timing_s/gen:12.418748388066888 - timing_s/reward:0.06932966038584709 - timing_s/old_log_prob:2.232461268082261 - timing_s/adv:0.5236291699111462 - timing_s/update_actor:15.485476253554225 - timing_s/step:30.817566089332104 - timing_s/stop_profile:0.000115908682346344 - timing_per_token_ms/adv:0.0037575197869552305 - timing_per_token_ms/gen:0.16709159194417458 - timing_per_token_ms/update_actor:0.1111225019091832 - perf/total_num_tokens:139355 - perf/time_per_step:30.817566089332104 - perf/throughput:565.2417504194123 (TaskRunner pid=2026458) Training Progress: 84%|████████▍ | 84/100 [59:18<09:42, 36.38s/it] (TaskRunner pid=2026458) step:85 - global_seqlen/min:16721 - global_seqlen/max:24217 - global_seqlen/minmax_diff:7496 - global_seqlen/balanced_min:19463 - global_seqlen/balanced_max:19465 - global_seqlen/mean:19464.25 - actor/entropy:0.23170782625675201 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5486323833465576 - training/rollout_probs_diff_mean:0.005258036311715841 - training/rollout_probs_diff_std:0.013953270390629768 - training/rollout_actor_probs_pearson_corr:0.998067319393158 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.75 - self_distillation/correct_sample_fraction:0.69921875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.75 - rollout_corr/rollout_is_mean:1.0003063678741455 - rollout_corr/rollout_is_ratio_fraction_high:2.5328639821964316e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.065727964392863e-05 - rollout_corr/rollout_is_max:2.629549741744995 - rollout_corr/rollout_is_min:0.32894784212112427 - rollout_corr/rollout_is_std:0.04162293300032616 - rollout_corr/rollout_is_eff_sample_size:0.9982714780308726 - rollout_corr/rollout_is_seq_mean:1.0004109144210815 - rollout_corr/rollout_is_seq_std:0.002749430714175105 - rollout_corr/rollout_is_seq_max:1.0109564065933228 - rollout_corr/rollout_is_seq_min:0.9916076064109802 - rollout_corr/rollout_is_seq_max_deviation:0.010956406593322754 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2771666692569852) - rollout_corr/training_log_ppl:np.float64(0.24135592477978207) - rollout_corr/kl:np.float64(0.0007578132510701607) - rollout_corr/k3_kl:np.float64(0.0010257670675173358) - rollout_corr/rollout_ppl:np.float64(1.2761707119643688) - rollout_corr/rollout_log_ppl:np.float64(0.24059811038023327) - rollout_corr/log_ppl_diff:np.float64(0.0007578143995488063) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024299477954627946) - rollout_corr/log_ppl_diff_max:np.float64(0.011833712458610535) - rollout_corr/log_ppl_diff_min:np.float64(-0.010344773530960083) - rollout_corr/ppl_ratio:np.float64(1.0007629161700606) - rollout_corr/chi2_token:np.float64(0.0026269492227584124) - rollout_corr/chi2_seq:np.float64(1.3403879939578474) - actor/pg_loss:np.float64(0.002229055855423212) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019035061723116087) - actor/ppo_kl:np.float64(0.00024953764273760726) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.75) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.75) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.69921875) - self_distillation/token_reweight_w_mean:np.float64(17317.446490210947) - self_distillation/token_reweight_w_std:np.float64(276465.0479241874) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9991332795470953) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03143878703122027) - self_distillation/empty_target_batch:np.float64(0.25) - actor/grad_norm:np.float64(0.2347664088010788) - perf/mfu/actor:np.float64(0.03632172645099713) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.27149963378906) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:1 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0004859932523686439 - critic/advantages/max:0.5 - critic/advantages/min:-0.875 - critic/returns/mean:0.0004859932523686439 - critic/returns/max:0.5 - critic/returns/min:-0.875 - response_length/mean:308.4453125 - response_length/max:777.0 - response_length/min:123.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:308.4453125 - response_length_non_aborted/max:777.0 - response_length_non_aborted/min:123.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:299.8125 - prompt_length/max:706.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014561228454113007 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4749425910413265) - timing_s/agent_loop/generate_sequences/max:np.float64(6.026074631139636) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2942245794984046) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.026074631139636) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:235 - timing_s/agent_loop/slowest/response_length:777 - timing_s/gen:12.095296228304505 - timing_s/reward:0.05645551718771458 - timing_s/old_log_prob:2.26230669952929 - timing_s/adv:0.4968310799449682 - timing_s/update_actor:15.393540350720286 - timing_s/step:30.395571315661073 - timing_s/stop_profile:0.00012073665857315063 - timing_per_token_ms/adv:0.0031906641660028527 - timing_per_token_ms/gen:0.15317869644011683 - timing_per_token_ms/update_actor:0.09885777997302932 - perf/total_num_tokens:155714 - perf/time_per_step:30.395571315661073 - perf/throughput:640.3646701640117 (TaskRunner pid=2026458) Training Progress: 85%|████████▌ | 85/100 [59:48<08:38, 34.60s/it] (TaskRunner pid=2026458) step:86 - global_seqlen/min:16265 - global_seqlen/max:20200 - global_seqlen/minmax_diff:3935 - global_seqlen/balanced_min:18647 - global_seqlen/balanced_max:18651 - global_seqlen/mean:18649.75 - actor/entropy:0.23041389882564545 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.898743212223053 - training/rollout_probs_diff_mean:0.005259583238512278 - training/rollout_probs_diff_std:0.014470859430730343 - training/rollout_actor_probs_pearson_corr:0.9979584813117981 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.734375 - self_distillation/correct_sample_fraction:0.5625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.734375 - rollout_corr/rollout_is_mean:0.9998511075973511 - rollout_corr/rollout_is_ratio_fraction_high:3.836807809420861e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.394679803634062e-05 - rollout_corr/rollout_is_max:2.1972851753234863 - rollout_corr/rollout_is_min:0.0777815580368042 - rollout_corr/rollout_is_std:0.04242711886763573 - rollout_corr/rollout_is_eff_sample_size:0.9982026989977762 - rollout_corr/rollout_is_seq_mean:0.9998696446418762 - rollout_corr/rollout_is_seq_std:0.0025317175313830376 - rollout_corr/rollout_is_seq_max:1.009598731994629 - rollout_corr/rollout_is_seq_min:0.9924889802932739 - rollout_corr/rollout_is_seq_max_deviation:0.009598731994628906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2799476818181574) - rollout_corr/training_log_ppl:np.float64(0.24274212954333052) - rollout_corr/kl:np.float64(0.0012324780372026112) - rollout_corr/k3_kl:np.float64(0.0011905511769896293) - rollout_corr/rollout_ppl:np.float64(1.2783302562311292) - rollout_corr/rollout_log_ppl:np.float64(0.24150965179433115) - rollout_corr/log_ppl_diff:np.float64(0.0012324777489993721) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024851207563187927) - rollout_corr/log_ppl_diff_max:np.float64(0.009878866374492645) - rollout_corr/log_ppl_diff_min:np.float64(-0.00684240460395813) - rollout_corr/ppl_ratio:np.float64(1.0012374119833112) - rollout_corr/chi2_token:np.float64(0.0022098186891525984) - rollout_corr/chi2_seq:np.float64(0.6068784659728408) - actor/pg_loss:np.float64(0.0033142786705866456) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009858607454589219) - actor/ppo_kl:np.float64(0.0001475137922213321) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.734375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.734375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.5625) - self_distillation/token_reweight_w_mean:np.float64(165894.66037644562) - self_distillation/token_reweight_w_std:np.float64(2346342.531521928) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0011869312729686) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07381428830558434) - self_distillation/empty_target_batch:np.float64(0.265625) - actor/grad_norm:np.float64(0.42134569585323334) - perf/mfu/actor:np.float64(0.0348608857845462) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1626739501953) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:1 - critic/score/mean:0.5625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0060413735918700695 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0060413735918700695 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:305.4296875 - response_length/max:825.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:305.4296875 - response_length_non_aborted/max:825.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.375 - prompt_length/max:503.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016151554882526398 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6213552672415972) - timing_s/agent_loop/generate_sequences/max:np.float64(6.171470927074552) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.300390396914736) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(6.171470927074552) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:825 - timing_s/gen:12.024075420573354 - timing_s/reward:0.05586143955588341 - timing_s/old_log_prob:2.3416189160197973 - timing_s/adv:0.5560324247926474 - timing_s/update_actor:15.314254153519869 - timing_s/step:30.382415380328894 - timing_s/stop_profile:0.0001287907361984253 - timing_per_token_ms/adv:0.0037268088365302976 - timing_per_token_ms/gen:0.15378022024009916 - timing_per_token_ms/update_actor:0.10264383003471808 - perf/total_num_tokens:149198 - perf/time_per_step:30.382415380328894 - perf/throughput:613.8336852597567 (TaskRunner pid=2026458) Training Progress: 86%|████████▌ | 86/100 [1:00:19<07:46, 33.35s/it] (TaskRunner pid=2026458) (TaskRunner pid=2026458) step:87 - global_seqlen/min:16648 - global_seqlen/max:20097 - global_seqlen/minmax_diff:3449 - global_seqlen/balanced_min:18341 - global_seqlen/balanced_max:18344 - global_seqlen/mean:18342.375 - actor/entropy:0.24005606770515442 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44461295008659363 - training/rollout_probs_diff_mean:0.005454737227410078 - training/rollout_probs_diff_std:0.014133700169622898 - training/rollout_actor_probs_pearson_corr:0.998093843460083 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.6796875 - self_distillation/correct_sample_fraction:0.53125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6796875 - rollout_corr/rollout_is_mean:0.9998968243598938 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.625958146993071e-05 - rollout_corr/rollout_is_max:1.91322922706604 - rollout_corr/rollout_is_min:0.3312951922416687 - rollout_corr/rollout_is_std:0.0426471121609211 - rollout_corr/rollout_is_eff_sample_size:0.998184050703313 - rollout_corr/rollout_is_seq_mean:0.999923050403595 - rollout_corr/rollout_is_seq_std:0.0029033836908638477 - rollout_corr/rollout_is_seq_max:1.0088640451431274 - rollout_corr/rollout_is_seq_min:0.990547776222229 - rollout_corr/rollout_is_seq_max_deviation:0.009452223777770996 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2908267816528678) - rollout_corr/training_log_ppl:np.float64(0.25219142163405195) - rollout_corr/kl:np.float64(0.0013515868227500505) - rollout_corr/k3_kl:np.float64(0.001383622248795291) - rollout_corr/rollout_ppl:np.float64(1.2890665112063289) - rollout_corr/rollout_log_ppl:np.float64(0.25083983279182576) - rollout_corr/log_ppl_diff:np.float64(0.0013515888422261924) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028635623457375914) - rollout_corr/log_ppl_diff_max:np.float64(0.012585937976837158) - rollout_corr/log_ppl_diff_min:np.float64(-0.009711101651191711) - rollout_corr/ppl_ratio:np.float64(1.0013586606364697) - rollout_corr/chi2_token:np.float64(0.0029732428956776857) - rollout_corr/chi2_seq:np.float64(0.6213931296952069) - actor/pg_loss:np.float64(0.005519538884982467) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011970178129558917) - actor/ppo_kl:np.float64(0.00032003364054844496) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6796875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6796875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53125) - self_distillation/token_reweight_w_mean:np.float64(104019.92419685167) - self_distillation/token_reweight_w_std:np.float64(1491533.3107083726) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000025857705623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07910635319421999) - self_distillation/empty_target_batch:np.float64(0.3203125) - actor/grad_norm:np.float64(0.44405730813741684) - perf/mfu/actor:np.float64(0.03427082030528879) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.2020378112793) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:1 - critic/score/mean:0.53125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009922783821821213 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009922783821821213 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:277.73046875 - response_length/max:642.0 - response_length/min:112.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:277.73046875 - response_length_non_aborted/max:642.0 - response_length_non_aborted/min:112.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:295.46875 - prompt_length/max:845.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011511892080307007 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3540627993643284) - timing_s/agent_loop/generate_sequences/max:np.float64(5.077815180644393) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.9762204922371893) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.077815180644393) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:172 - timing_s/agent_loop/slowest/response_length:642 - timing_s/gen:11.154959298670292 - timing_s/reward:0.057516856119036674 - timing_s/old_log_prob:2.265875956043601 - timing_s/adv:0.5384714491665363 - timing_s/update_actor:15.248491061851382 - timing_s/step:29.354633087292314 - timing_s/stop_profile:0.0001263357698917389 - timing_per_token_ms/adv:0.003669586470989555 - timing_per_token_ms/gen:0.15689333603384426 - timing_per_token_ms/update_actor:0.10391573516141846 - perf/total_num_tokens:146739 - perf/time_per_step:29.354633087292314 - perf/throughput:624.8545142926844 (TaskRunner pid=2026458) Training Progress: 87%|████████▋ | 87/100 [1:00:48<06:58, 32.18s/it] (TaskRunner pid=2026458) step:88 - global_seqlen/min:14981 - global_seqlen/max:20248 - global_seqlen/minmax_diff:5267 - global_seqlen/balanced_min:17171 - global_seqlen/balanced_max:17176 - global_seqlen/mean:17174.125 - actor/entropy:0.21586236357688904 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5279986262321472 - training/rollout_probs_diff_mean:0.005149980541318655 - training/rollout_probs_diff_std:0.014039847999811172 - training/rollout_actor_probs_pearson_corr:0.9979734420776367 - self_distillation/success_group_fraction:0.75 - self_distillation/success_sample_fraction:0.7421875 - self_distillation/correct_sample_fraction:0.57421875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7421875 - rollout_corr/rollout_is_mean:0.9998298287391663 - rollout_corr/rollout_is_ratio_fraction_high:2.9164297302486375e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.4582148651243187e-05 - rollout_corr/rollout_is_max:2.4087512493133545 - rollout_corr/rollout_is_min:0.3157358169555664 - rollout_corr/rollout_is_std:0.04135211929678917 - rollout_corr/rollout_is_eff_sample_size:0.998292149229813 - rollout_corr/rollout_is_seq_mean:0.9997440576553345 - rollout_corr/rollout_is_seq_std:0.0026299559976905584 - rollout_corr/rollout_is_seq_max:1.0093685388565063 - rollout_corr/rollout_is_seq_min:0.9928037524223328 - rollout_corr/rollout_is_seq_max_deviation:0.009368538856506348 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.263124991208315) - rollout_corr/training_log_ppl:np.float64(0.23035154843819328) - rollout_corr/kl:np.float64(0.0012507079140435806) - rollout_corr/k3_kl:np.float64(0.0012822421521718752) - rollout_corr/rollout_ppl:np.float64(1.2614940297789872) - rollout_corr/rollout_log_ppl:np.float64(0.22910083959868643) - rollout_corr/log_ppl_diff:np.float64(0.0012507088395068422) - rollout_corr/log_ppl_abs_diff:np.float64(0.0027966901106992736) - rollout_corr/log_ppl_diff_max:np.float64(0.014727145433425903) - rollout_corr/log_ppl_diff_min:np.float64(-0.014272093772888184) - rollout_corr/ppl_ratio:np.float64(1.0012579679023474) - rollout_corr/chi2_token:np.float64(0.002700346987694502) - rollout_corr/chi2_seq:np.float64(0.8957935450598598) - actor/pg_loss:np.float64(0.0045979529386386275) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008685116490596556) - actor/ppo_kl:np.float64(0.00010619230280539682) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7421875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7421875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.57421875) - self_distillation/token_reweight_w_mean:np.float64(126031.5825594815) - self_distillation/token_reweight_w_std:np.float64(1914441.9060592402) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0010992304887623) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07210871722782031) - self_distillation/empty_target_batch:np.float64(0.2578125) - actor/grad_norm:np.float64(0.4531179219484329) - perf/mfu/actor:np.float64(0.032191482757026275) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.15955352783203) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:1 - critic/score/mean:0.57421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.57421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0014600376598536968 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0014600376598536968 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:267.87890625 - response_length/max:807.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:267.87890625 - response_length_non_aborted/max:807.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.8125 - prompt_length/max:482.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012998655438423157 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3904380090534687) - timing_s/agent_loop/generate_sequences/max:np.float64(5.8301611468195915) - timing_s/agent_loop/generate_sequences/mean:np.float64(2.8218457526090788) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.8301611468195915) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:807 - timing_s/gen:11.686277892440557 - timing_s/reward:0.054150139912962914 - timing_s/old_log_prob:2.385304968804121 - timing_s/adv:0.4924223218113184 - timing_s/update_actor:15.133386766538024 - timing_s/step:29.843838276341558 - timing_s/stop_profile:5.164928734302521e-05 - timing_per_token_ms/adv:0.003584042286079483 - timing_per_token_ms/gen:0.170411040034422 - timing_per_token_ms/update_actor:0.11014670883187662 - perf/total_num_tokens:137393 - perf/time_per_step:29.843838276341558 - perf/throughput:575.466360626094 (TaskRunner pid=2026458) Training Progress: 88%|████████▊ | 88/100 [1:01:18<06:17, 31.49s/it] (TaskRunner pid=2026458) step:89 - global_seqlen/min:16152 - global_seqlen/max:22412 - global_seqlen/minmax_diff:6260 - global_seqlen/balanced_min:18887 - global_seqlen/balanced_max:18895 - global_seqlen/mean:18893.375 - actor/entropy:0.22386007010936737 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2151239514350891 - training/rollout_probs_diff_mean:0.005173899233341217 - training/rollout_probs_diff_std:0.013607512228190899 - training/rollout_actor_probs_pearson_corr:0.9981110692024231 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.83203125 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.83203125 - rollout_corr/rollout_is_mean:1.0000429153442383 - rollout_corr/rollout_is_ratio_fraction_high:2.6122277631657198e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.83668365329504e-05 - rollout_corr/rollout_is_max:2.51365327835083 - rollout_corr/rollout_is_min:0.1054598018527031 - rollout_corr/rollout_is_std:0.04179442301392555 - rollout_corr/rollout_is_eff_sample_size:0.9982565097660324 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0027215098962187767 - rollout_corr/rollout_is_seq_max:1.0083972215652466 - rollout_corr/rollout_is_seq_min:0.9915516376495361 - rollout_corr/rollout_is_seq_max_deviation:0.008448362350463867 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2823212435469031) - rollout_corr/training_log_ppl:np.float64(0.24520724279864226) - rollout_corr/kl:np.float64(0.0008697826999508607) - rollout_corr/k3_kl:np.float64(0.0010809666277964425) - rollout_corr/rollout_ppl:np.float64(1.2811899655498564) - rollout_corr/rollout_log_ppl:np.float64(0.24433745855640154) - rollout_corr/log_ppl_diff:np.float64(0.000869784242240712) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022608327853959054) - rollout_corr/log_ppl_diff_max:np.float64(0.010239839553833008) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071149468421936035) - rollout_corr/ppl_ratio:np.float64(1.0008741298224777) - rollout_corr/chi2_token:np.float64(0.0025997732300311327) - rollout_corr/chi2_seq:np.float64(0.5740627928171307) - actor/pg_loss:np.float64(0.0036603367188945413) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008643267137813382) - actor/ppo_kl:np.float64(-9.36468737013918e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.83203125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.83203125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(95742.81214469112) - self_distillation/token_reweight_w_std:np.float64(1397763.4353546314) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0014505761209875) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07612263548071496) - self_distillation/empty_target_batch:np.float64(0.16796875) - actor/grad_norm:np.float64(0.43359871953725815) - perf/mfu/actor:np.float64(0.035258064824575705) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.1541633605957) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.00761464424431324 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.00761464424431324 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:299.07421875 - response_length/max:1025.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:299.07421875 - response_length_non_aborted/max:1025.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:291.34375 - prompt_length/max:720.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.2774325013160706e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3900669310241938) - timing_s/agent_loop/generate_sequences/max:np.float64(7.259778779000044) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.168158708969713) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(7.259778779000044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:1025 - timing_s/gen:12.976460268720984 - timing_s/reward:0.055463384836912155 - timing_s/old_log_prob:2.4733975417912006 - timing_s/adv:0.5145350992679596 - timing_s/update_actor:15.292766243219376 - timing_s/step:31.39907766878605 - timing_s/stop_profile:4.6625733375549316e-05 - timing_per_token_ms/adv:0.003404203188074918 - timing_per_token_ms/gen:0.16948735379649418 - timing_per_token_ms/update_actor:0.10117809975202535 - perf/total_num_tokens:151147 - perf/time_per_step:31.39907766878605 - perf/throughput:601.7175153772744 (TaskRunner pid=2026458) Training Progress: 89%|████████▉ | 89/100 [1:01:50<05:46, 31.47s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a chlorine atom, a trifluoromethyl group (C(F)(F)F), and a specific aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A contains a structure with a chlorine atom, a trifluoromethyl group, and the correct aromatic ring structure. The formula is "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1", which matches the required components of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B contains a chlorine atom but does not have the correct trifluoromethyl group or the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the correct answer is Option A, as it contains all the necessary components of the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2026458) step:90 - global_seqlen/min:14894 - global_seqlen/max:19933 - global_seqlen/minmax_diff:5039 - global_seqlen/balanced_min:18191 - global_seqlen/balanced_max:18194 - global_seqlen/mean:18192.75 - actor/entropy:0.21710193157196045 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3145221471786499 - training/rollout_probs_diff_mean:0.004972291644662619 - training/rollout_probs_diff_std:0.013361528515815735 - training/rollout_actor_probs_pearson_corr:0.9982044100761414 - self_distillation/success_group_fraction:0.6875 - self_distillation/success_sample_fraction:0.67578125 - self_distillation/correct_sample_fraction:0.53515625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.67578125 - rollout_corr/rollout_is_mean:1.0000026226043701 - rollout_corr/rollout_is_ratio_fraction_high:2.6879552024183795e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.0319329855265096e-05 - rollout_corr/rollout_is_max:3.3088538646698 - rollout_corr/rollout_is_min:0.46167826652526855 - rollout_corr/rollout_is_std:0.041136063635349274 - rollout_corr/rollout_is_eff_sample_size:0.9983105639498434 - rollout_corr/rollout_is_seq_mean:1.0000332593917847 - rollout_corr/rollout_is_seq_std:0.0027625819202512503 - rollout_corr/rollout_is_seq_max:1.0086408853530884 - rollout_corr/rollout_is_seq_min:0.9904849529266357 - rollout_corr/rollout_is_seq_max_deviation:0.009515047073364258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2710709455423057) - rollout_corr/training_log_ppl:np.float64(0.23564479569904506) - rollout_corr/kl:np.float64(0.0014168473381213076) - rollout_corr/k3_kl:np.float64(0.0014046167735841664) - rollout_corr/rollout_ppl:np.float64(1.2692075530067086) - rollout_corr/rollout_log_ppl:np.float64(0.2342279483564198) - rollout_corr/log_ppl_diff:np.float64(0.0014168473426252604) - rollout_corr/log_ppl_abs_diff:np.float64(0.0028690413455478847) - rollout_corr/log_ppl_diff_max:np.float64(0.03278002142906189) - rollout_corr/log_ppl_diff_min:np.float64(-0.008600443601608276) - rollout_corr/ppl_ratio:np.float64(1.0014270655810833) - rollout_corr/chi2_token:np.float64(0.002751560416072607) - rollout_corr/chi2_seq:np.float64(0.7767476551234722) - actor/pg_loss:np.float64(0.0019601467065513134) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009453206475882325) - actor/ppo_kl:np.float64(0.0005355499455532708) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.67578125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.67578125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.53515625) - self_distillation/token_reweight_w_mean:np.float64(99385.55734024942) - self_distillation/token_reweight_w_std:np.float64(1449312.0253919773) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0012841576244682) - self_distillation/token_reweight_w_clip_frac:np.float64(0.045599023724207655) - self_distillation/empty_target_batch:np.float64(0.32421875) - actor/grad_norm:np.float64(0.30510950461030006) - perf/mfu/actor:np.float64(0.03390671346645702) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.11214065551758) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/reward/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.07254164422675807) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8122666666666666) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.05053109084853488) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7015285714285714) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05338104590757411) - val-aux/sciknoweval/score/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/score/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/score/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/score/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.07254164422675807) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8122666666666666) - val-aux/sciknoweval/score/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.05053109084853488) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7015285714285714) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05338104590757411) - val-core/sciknoweval/acc/mean@16:np.float64(0.6821428571428572) - val-aux/sciknoweval/acc/std@16:np.float64(0.1287410217939347) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7336095238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.0988808144488825) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6304571428571429) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.11689339649607558) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6818380952380951) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.127482055226586) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.771042857142857) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.06672262251716281) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5835476190476191) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.09690360977896326) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6921333333333333) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.10110972005167294) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7954380952380954) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.04301822225190089) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5451904761904762) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07359385049232874) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6976380952380952) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.07254164422675807) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8122666666666666) - val-core/sciknoweval/acc/best@16/std:np.float64(0.027241398706241732) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5151047619047618) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.05053109084853488) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7015285714285714) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05338104590757411) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0003677473252630062) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9995190476190476) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.001434898686445427) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997809523809524) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0009975480142912052) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9990666666666667) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0018903262505010432) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999142857142858) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0006331005658902953) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9981523809523809) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0023204503433425093) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9999714285714286) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0003677473252630062) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9970095238095239) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0023016113846578) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:1 - critic/score/mean:0.53515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.53515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002840832807123661 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002840832807123661 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:290.6484375 - response_length/max:721.0 - response_length/min:94.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:290.6484375 - response_length_non_aborted/max:721.0 - response_length_non_aborted/min:94.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.875 - prompt_length/max:621.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.825903594493866e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7425885982811451) - timing_s/agent_loop/generate_sequences/max:np.float64(5.7153176963329315) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0369765825307695) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.7153176963329315) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:721 - timing_s/gen:11.199985072016716 - timing_s/reward:0.05536814592778683 - timing_s/old_log_prob:2.218005755916238 - timing_s/adv:0.4808800220489502 - timing_s/update_actor:15.297142412513494 - timing_s/step:29.337590511888266 - timing_s/testing:111.78063779324293 - timing_s/save_checkpoint:6.6935704834759235 - timing_s/stop_profile:0.00011844001710414886 - timing_per_token_ms/adv:0.003304063583357039 - timing_per_token_ms/gen:0.15052529462700207 - timing_per_token_ms/update_actor:0.10510465990925984 - perf/total_num_tokens:145542 - perf/time_per_step:29.337590511888266 - perf/throughput:620.1173880529786 (TaskRunner pid=2026458) Training Progress: 90%|█████████ | 90/100 [1:04:17<11:03, 66.39s/it] (TaskRunner pid=2026458) step:91 - global_seqlen/min:15865 - global_seqlen/max:25244 - global_seqlen/minmax_diff:9379 - global_seqlen/balanced_min:18811 - global_seqlen/balanced_max:18818 - global_seqlen/mean:18815.125 - actor/entropy:0.2314702272415161 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21055352687835693 - training/rollout_probs_diff_mean:0.005051769316196442 - training/rollout_probs_diff_std:0.013311025686562061 - training/rollout_actor_probs_pearson_corr:0.9982390999794006 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.60546875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:0.9999205470085144 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.47056513116695e-05 - rollout_corr/rollout_is_max:1.7587093114852905 - rollout_corr/rollout_is_min:0.23516227304935455 - rollout_corr/rollout_is_std:0.039755210280418396 - rollout_corr/rollout_is_eff_sample_size:0.9984218983709217 - rollout_corr/rollout_is_seq_mean:0.9999246001243591 - rollout_corr/rollout_is_seq_std:0.002499330323189497 - rollout_corr/rollout_is_seq_max:1.0098474025726318 - rollout_corr/rollout_is_seq_min:0.9917200207710266 - rollout_corr/rollout_is_seq_max_deviation:0.009847402572631836 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2789378436282277) - rollout_corr/training_log_ppl:np.float64(0.24201013351557776) - rollout_corr/kl:np.float64(0.0009981181076650358) - rollout_corr/k3_kl:np.float64(0.0010161043154539584) - rollout_corr/rollout_ppl:np.float64(1.2776579768396914) - rollout_corr/rollout_log_ppl:np.float64(0.2410120142158121) - rollout_corr/log_ppl_diff:np.float64(0.0009981192997656763) - rollout_corr/log_ppl_abs_diff:np.float64(0.002134583191946149) - rollout_corr/log_ppl_diff_max:np.float64(0.00839272141456604) - rollout_corr/log_ppl_diff_min:np.float64(-0.010266929864883423) - rollout_corr/ppl_ratio:np.float64(1.0010020262561738) - rollout_corr/chi2_token:np.float64(0.0021203176584094763) - rollout_corr/chi2_seq:np.float64(0.651163290720433) - actor/pg_loss:np.float64(0.002993874717503786) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000568103197565506) - actor/ppo_kl:np.float64(6.333214638765128e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.60546875) - self_distillation/token_reweight_w_mean:np.float64(82354.48971123807) - self_distillation/token_reweight_w_std:np.float64(1252927.9291606762) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0009026953484863) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06239277429995127) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.33179403841495514) - perf/mfu/actor:np.float64(0.034982824171721626) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.52531814575195) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:1 - critic/score/mean:0.60546875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.60546875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.002352050505578518 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.002352050505578518 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:301.84765625 - response_length/max:674.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.84765625 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:286.125 - prompt_length/max:1009.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.271526217460632e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2008809298276901) - timing_s/agent_loop/generate_sequences/max:np.float64(5.367214092984796) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2526293304326828) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.367214092984796) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:208 - timing_s/agent_loop/slowest/response_length:645 - timing_s/gen:11.187385501340032 - timing_s/reward:0.05164775624871254 - timing_s/old_log_prob:2.4035099502652884 - timing_s/adv:0.5490099098533392 - timing_s/update_actor:15.261925529688597 - timing_s/step:29.56569225527346 - timing_s/stop_profile:0.00012163445353507996 - timing_per_token_ms/adv:0.0036473974385855742 - timing_per_token_ms/gen:0.14477741903821556 - timing_per_token_ms/update_actor:0.10139399505509927 - perf/total_num_tokens:150521 - perf/time_per_step:29.56569225527346 - perf/throughput:636.3837124985313 (TaskRunner pid=2026458) Training Progress: 91%|█████████ | 91/100 [1:04:47<08:18, 55.36s/it] (TaskRunner pid=2026458) step:92 - global_seqlen/min:16281 - global_seqlen/max:21475 - global_seqlen/minmax_diff:5194 - global_seqlen/balanced_min:18650 - global_seqlen/balanced_max:18651 - global_seqlen/mean:18650.625 - actor/entropy:0.22619383037090302 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.005189774092286825 - training/rollout_probs_diff_std:0.013963738456368446 - training/rollout_actor_probs_pearson_corr:0.9980257153511047 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.7109375 - self_distillation/correct_sample_fraction:0.625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.7109375 - rollout_corr/rollout_is_mean:0.9999138116836548 - rollout_corr/rollout_is_ratio_fraction_high:1.3226638657215517e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00011903974518645555 - rollout_corr/rollout_is_max:2.0864930152893066 - rollout_corr/rollout_is_min:0.0921701192855835 - rollout_corr/rollout_is_std:0.04230894520878792 - rollout_corr/rollout_is_eff_sample_size:0.9982129142824776 - rollout_corr/rollout_is_seq_mean:0.9999446272850037 - rollout_corr/rollout_is_seq_std:0.0027264179661870003 - rollout_corr/rollout_is_seq_max:1.0081113576889038 - rollout_corr/rollout_is_seq_min:0.9917219281196594 - rollout_corr/rollout_is_seq_max_deviation:0.008278071880340576 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2724861456081271) - rollout_corr/training_log_ppl:np.float64(0.2372018521418795) - rollout_corr/kl:np.float64(0.0009739822945320498) - rollout_corr/k3_kl:np.float64(0.0010123744499423992) - rollout_corr/rollout_ppl:np.float64(1.2711997576989233) - rollout_corr/rollout_log_ppl:np.float64(0.23622786854684819) - rollout_corr/log_ppl_diff:np.float64(0.0009739835950313136) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024230604904005304) - rollout_corr/log_ppl_diff_max:np.float64(0.012229472398757935) - rollout_corr/log_ppl_diff_min:np.float64(-0.010651081800460815) - rollout_corr/ppl_ratio:np.float64(1.0009791306219995) - rollout_corr/chi2_token:np.float64(0.0021006297320127487) - rollout_corr/chi2_seq:np.float64(0.7930810896214098) - actor/pg_loss:np.float64(0.002591841504909098) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004880208070972003) - actor/ppo_kl:np.float64(-2.9975729340492308e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.7109375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.7109375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.625) - self_distillation/token_reweight_w_mean:np.float64(72610.33176031034) - self_distillation/token_reweight_w_std:np.float64(1092734.639293266) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9988934537395835) - self_distillation/token_reweight_w_clip_frac:np.float64(0.06326231267303228) - self_distillation/empty_target_batch:np.float64(0.2890625) - actor/grad_norm:np.float64(0.3366621397435665) - perf/mfu/actor:np.float64(0.03433711138530011) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.60525131225586) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:1 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0006365319713950157 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0006365319713950157 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:295.33203125 - response_length/max:674.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:295.33203125 - response_length_non_aborted/max:674.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:287.5 - prompt_length/max:436.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.356330752372742e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4242578689008951) - timing_s/agent_loop/generate_sequences/max:np.float64(5.455314517021179) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2662257757328916) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.455314517021179) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:193 - timing_s/agent_loop/slowest/response_length:674 - timing_s/gen:11.507964111864567 - timing_s/reward:0.056888729333877563 - timing_s/old_log_prob:2.3034613393247128 - timing_s/adv:0.4925750717520714 - timing_s/update_actor:15.52449726127088 - timing_s/step:29.975734889507294 - timing_s/stop_profile:0.000132659450173378 - timing_per_token_ms/adv:0.0033013308652663875 - timing_per_token_ms/gen:0.15221168060134338 - timing_per_token_ms/update_actor:0.10404810335626072 - perf/total_num_tokens:149205 - perf/time_per_step:29.975734889507294 - perf/throughput:622.1907509106129 (TaskRunner pid=2026458) Training Progress: 92%|█████████▏| 92/100 [1:05:17<06:22, 47.76s/it] (TaskRunner pid=2026458) step:93 - global_seqlen/min:15058 - global_seqlen/max:22608 - global_seqlen/minmax_diff:7550 - global_seqlen/balanced_min:18025 - global_seqlen/balanced_max:18028 - global_seqlen/mean:18026.625 - actor/entropy:0.20888566970825195 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24053943157196045 - training/rollout_probs_diff_mean:0.004918403923511505 - training/rollout_probs_diff_std:0.013119952753186226 - training/rollout_actor_probs_pearson_corr:0.9981909394264221 - self_distillation/success_group_fraction:1.0 - self_distillation/success_sample_fraction:0.98828125 - self_distillation/correct_sample_fraction:0.87890625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.98828125 - rollout_corr/rollout_is_mean:0.9998151659965515 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.263365528662689e-05 - rollout_corr/rollout_is_max:1.6370371580123901 - rollout_corr/rollout_is_min:0.2820625901222229 - rollout_corr/rollout_is_std:0.03976345434784889 - rollout_corr/rollout_is_eff_sample_size:0.9984207100389592 - rollout_corr/rollout_is_seq_mean:0.9998314380645752 - rollout_corr/rollout_is_seq_std:0.0023552256170660257 - rollout_corr/rollout_is_seq_max:1.0079485177993774 - rollout_corr/rollout_is_seq_min:0.9893273115158081 - rollout_corr/rollout_is_seq_max_deviation:0.010672688484191895 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2420122399926186) - rollout_corr/training_log_ppl:np.float64(0.2130839267920237) - rollout_corr/kl:np.float64(0.0009293865148691793) - rollout_corr/k3_kl:np.float64(0.000889873810649533) - rollout_corr/rollout_ppl:np.float64(1.2408293401822448) - rollout_corr/rollout_log_ppl:np.float64(0.2121545401896583) - rollout_corr/log_ppl_diff:np.float64(0.0009293866023654118) - rollout_corr/log_ppl_abs_diff:np.float64(0.00208455762185622) - rollout_corr/log_ppl_diff_max:np.float64(0.008339852094650269) - rollout_corr/log_ppl_diff_min:np.float64(-0.005493566393852234) - rollout_corr/ppl_ratio:np.float64(1.0009328972082585) - rollout_corr/chi2_token:np.float64(0.00168294133618474) - rollout_corr/chi2_seq:np.float64(0.3678914944175631) - actor/pg_loss:np.float64(0.00139771425165236) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002747084317888948) - actor/ppo_kl:np.float64(-7.94472919167788e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.98828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.98828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.87890625) - self_distillation/token_reweight_w_mean:np.float64(72035.54537807149) - self_distillation/token_reweight_w_std:np.float64(1075218.813784783) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.001510763540864) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03214944276260212) - self_distillation/empty_target_batch:np.float64(0.01171875) - actor/grad_norm:np.float64(0.2579457201063633) - perf/mfu/actor:np.float64(0.03364477703539987) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.39699172973633) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:1 - critic/score/mean:0.87890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.87890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0036366567946970463 - critic/advantages/max:0.875 - critic/advantages/min:-0.625 - critic/returns/mean:0.0036366567946970463 - critic/returns/max:0.875 - critic/returns/min:-0.625 - response_length/mean:296.86328125 - response_length/max:640.0 - response_length/min:140.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:296.86328125 - response_length_non_aborted/max:640.0 - response_length_non_aborted/min:140.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:709.0 - prompt_length/min:167.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017446838319301605 - timing_s/agent_loop/generate_sequences/min:np.float64(1.6789815910160542) - timing_s/agent_loop/generate_sequences/max:np.float64(5.267441829666495) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.250441880845756) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.267441829666495) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:640 - timing_s/gen:11.287975236773491 - timing_s/reward:0.0554285254329443 - timing_s/old_log_prob:2.2384933289140463 - timing_s/adv:0.5122805014252663 - timing_s/update_actor:15.286979779601097 - timing_s/step:29.47672649472952 - timing_s/stop_profile:0.00014898553490638733 - timing_per_token_ms/adv:0.003552249113639313 - timing_per_token_ms/gen:0.14853185305700872 - timing_per_token_ms/update_actor:0.106002786015138 - perf/total_num_tokens:144213 - perf/time_per_step:29.47672649472952 - perf/throughput:611.5545090538865 (TaskRunner pid=2026458) Training Progress: 93%|█████████▎| 93/100 [1:05:47<04:56, 42.29s/it] (TaskRunner pid=2026458) step:94 - global_seqlen/min:15786 - global_seqlen/max:20010 - global_seqlen/minmax_diff:4224 - global_seqlen/balanced_min:17735 - global_seqlen/balanced_max:17737 - global_seqlen/mean:17736.0 - actor/entropy:0.22735926508903503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311311483383179 - training/rollout_probs_diff_mean:0.005040409974753857 - training/rollout_probs_diff_std:0.013486634008586407 - training/rollout_actor_probs_pearson_corr:0.9981728792190552 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.84375 - self_distillation/correct_sample_fraction:0.73046875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.84375 - rollout_corr/rollout_is_mean:0.9998102784156799 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0001036699177348055 - rollout_corr/rollout_is_max:1.7523070573806763 - rollout_corr/rollout_is_min:0.05793754756450653 - rollout_corr/rollout_is_std:0.03992203250527382 - rollout_corr/rollout_is_eff_sample_size:0.9984081138940653 - rollout_corr/rollout_is_seq_mean:0.9999222159385681 - rollout_corr/rollout_is_seq_std:0.002507504541426897 - rollout_corr/rollout_is_seq_max:1.0082566738128662 - rollout_corr/rollout_is_seq_min:0.9931005835533142 - rollout_corr/rollout_is_seq_max_deviation:0.008256673812866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2725239442661405) - rollout_corr/training_log_ppl:np.float64(0.23770636528206524) - rollout_corr/kl:np.float64(0.001132996877289294) - rollout_corr/k3_kl:np.float64(0.0012387144401486694) - rollout_corr/rollout_ppl:np.float64(1.2710481900721788) - rollout_corr/rollout_log_ppl:np.float64(0.23657336641917937) - rollout_corr/log_ppl_diff:np.float64(0.00113299886288587) - rollout_corr/log_ppl_abs_diff:np.float64(0.002540186673286371) - rollout_corr/log_ppl_diff_max:np.float64(0.02026122808456421) - rollout_corr/log_ppl_diff_min:np.float64(-0.015430659055709839) - rollout_corr/ppl_ratio:np.float64(1.0011393770109862) - rollout_corr/chi2_token:np.float64(0.002516603097319603) - rollout_corr/chi2_seq:np.float64(0.9786048375535756) - actor/pg_loss:np.float64(0.002404728322289884) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001062956328951259) - actor/ppo_kl:np.float64(0.0001939034154609942) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.84375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.84375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.73046875) - self_distillation/token_reweight_w_mean:np.float64(96479.46470297547) - self_distillation/token_reweight_w_std:np.float64(1265708.629157153) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9990413470659405) - self_distillation/token_reweight_w_clip_frac:np.float64(0.054301650350680575) - self_distillation/empty_target_batch:np.float64(0.15625) - actor/grad_norm:np.float64(0.35001390427351) - perf/mfu/actor:np.float64(0.032954674991088666) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.4532012939453) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:1 - critic/score/mean:0.73046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00508630508556962 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00508630508556962 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:301.4375 - response_length/max:631.0 - response_length/min:135.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:301.4375 - response_length_non_aborted/max:631.0 - response_length_non_aborted/min:135.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.8125 - prompt_length/max:384.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017067231237888336 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4371750131249428) - timing_s/agent_loop/generate_sequences/max:np.float64(5.477192688733339) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2628613221095293) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.477192688733339) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:221 - timing_s/agent_loop/slowest/response_length:631 - timing_s/gen:11.104732798412442 - timing_s/reward:0.05309521034359932 - timing_s/old_log_prob:2.3046134896576405 - timing_s/adv:0.4992757458239794 - timing_s/update_actor:15.339771771803498 - timing_s/step:29.391625313088298 - timing_s/stop_profile:0.000131336972117424 - timing_per_token_ms/adv:0.0035188017719890293 - timing_per_token_ms/gen:0.14390333815068995 - timing_per_token_ms/update_actor:0.10811183307822718 - perf/total_num_tokens:141888 - perf/time_per_step:29.391625313088298 - perf/throughput:603.4371971971906 (TaskRunner pid=2026458) Training Progress: 94%|█████████▍| 94/100 [1:06:16<03:50, 38.44s/it] (TaskRunner pid=2026458) step:95 - global_seqlen/min:14931 - global_seqlen/max:21336 - global_seqlen/minmax_diff:6405 - global_seqlen/balanced_min:17803 - global_seqlen/balanced_max:17812 - global_seqlen/mean:17810.75 - actor/entropy:0.22719933092594147 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3546398878097534 - training/rollout_probs_diff_mean:0.004889350384473801 - training/rollout_probs_diff_std:0.012968857772648335 - training/rollout_actor_probs_pearson_corr:0.9983217716217041 - self_distillation/success_group_fraction:0.84375 - self_distillation/success_sample_fraction:0.828125 - self_distillation/correct_sample_fraction:0.6640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.828125 - rollout_corr/rollout_is_mean:0.9999182224273682 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3313806448422838e-05 - rollout_corr/rollout_is_max:1.646693468093872 - rollout_corr/rollout_is_min:0.012675890699028969 - rollout_corr/rollout_is_std:0.03914937749505043 - rollout_corr/rollout_is_eff_sample_size:0.9984694339690914 - rollout_corr/rollout_is_seq_mean:0.9998860955238342 - rollout_corr/rollout_is_seq_std:0.002519140485674143 - rollout_corr/rollout_is_seq_max:1.0078048706054688 - rollout_corr/rollout_is_seq_min:0.993129551410675 - rollout_corr/rollout_is_seq_max_deviation:0.00780487060546875 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.269720992539078) - rollout_corr/training_log_ppl:np.float64(0.23537427955307066) - rollout_corr/kl:np.float64(0.000860856562256096) - rollout_corr/k3_kl:np.float64(0.0010757347620256041) - rollout_corr/rollout_ppl:np.float64(1.2686140895821154) - rollout_corr/rollout_log_ppl:np.float64(0.2345134206698276) - rollout_corr/log_ppl_diff:np.float64(0.0008608588832430542) - rollout_corr/log_ppl_abs_diff:np.float64(0.0024572820402681828) - rollout_corr/log_ppl_diff_max:np.float64(0.022655338048934937) - rollout_corr/log_ppl_diff_min:np.float64(-0.008849263191223145) - rollout_corr/ppl_ratio:np.float64(1.0008665048517287) - rollout_corr/chi2_token:np.float64(0.002582071116194129) - rollout_corr/chi2_seq:np.float64(1.1929384064860642) - actor/pg_loss:np.float64(0.0030269279377534986) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.001186676310680923) - actor/ppo_kl:np.float64(-0.00011882417348374474) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.828125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.828125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6640625) - self_distillation/token_reweight_w_mean:np.float64(142327.77909193072) - self_distillation/token_reweight_w_std:np.float64(1811469.8086368854) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0001867031678557) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07137267180951312) - self_distillation/empty_target_batch:np.float64(0.171875) - actor/grad_norm:np.float64(0.4086311012506485) - perf/mfu/actor:np.float64(0.03299815181672398) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.32865142822266) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007821861072443426 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007821861072443426 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.3984375 - response_length/max:742.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.3984375 - response_length_non_aborted/max:742.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.1875 - prompt_length/max:646.0 - prompt_length/min:159.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011094845831394196 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2119843941181898) - timing_s/agent_loop/generate_sequences/max:np.float64(5.851273892447352) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.2298050526515) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.851273892447352) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:411 - timing_s/agent_loop/slowest/response_length:742 - timing_s/gen:11.692811168730259 - timing_s/reward:0.051967186853289604 - timing_s/old_log_prob:2.230885935947299 - timing_s/adv:0.5060996375977993 - timing_s/update_actor:15.389210676774383 - timing_s/step:29.97092897631228 - timing_s/stop_profile:0.0001416075974702835 - timing_per_token_ms/adv:0.003551925365283602 - timing_per_token_ms/gen:0.15567582437398828 - timing_per_token_ms/update_actor:0.10800507191425392 - perf/total_num_tokens:142486 - perf/time_per_step:29.97092897631228 - perf/throughput:594.2675321834984 (TaskRunner pid=2026458) Training Progress: 95%|█████████▌| 95/100 [1:06:46<02:59, 35.91s/it] (TaskRunner pid=2026458) step:96 - global_seqlen/min:15504 - global_seqlen/max:21947 - global_seqlen/minmax_diff:6443 - global_seqlen/balanced_min:17847 - global_seqlen/balanced_max:17850 - global_seqlen/mean:17848.25 - actor/entropy:0.24302901327610016 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7704145312309265 - training/rollout_probs_diff_mean:0.0050596678629517555 - training/rollout_probs_diff_std:0.013549817726016045 - training/rollout_actor_probs_pearson_corr:0.9982749223709106 - self_distillation/success_group_fraction:0.78125 - self_distillation/success_sample_fraction:0.76953125 - self_distillation/correct_sample_fraction:0.6015625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.76953125 - rollout_corr/rollout_is_mean:1.0000941753387451 - rollout_corr/rollout_is_ratio_fraction_high:4.025656744488515e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.709428271278739e-05 - rollout_corr/rollout_is_max:2.5671136379241943 - rollout_corr/rollout_is_min:0.13419120013713837 - rollout_corr/rollout_is_std:0.039693690836429596 - rollout_corr/rollout_is_eff_sample_size:0.9984272458997584 - rollout_corr/rollout_is_seq_mean:1.000090479850769 - rollout_corr/rollout_is_seq_std:0.0026221859734505415 - rollout_corr/rollout_is_seq_max:1.009491205215454 - rollout_corr/rollout_is_seq_min:0.9908090233802795 - rollout_corr/rollout_is_seq_max_deviation:0.009491205215454102 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2904391190968454) - rollout_corr/training_log_ppl:np.float64(0.2496467249293346) - rollout_corr/kl:np.float64(0.0008864650635374005) - rollout_corr/k3_kl:np.float64(0.0009545528546368587) - rollout_corr/rollout_ppl:np.float64(1.2892436645925045) - rollout_corr/rollout_log_ppl:np.float64(0.24876026091806125) - rollout_corr/log_ppl_diff:np.float64(0.0008864640112733468) - rollout_corr/log_ppl_abs_diff:np.float64(0.0021889270428800955) - rollout_corr/log_ppl_diff_max:np.float64(0.013020426034927368) - rollout_corr/log_ppl_diff_min:np.float64(-0.006367728114128113) - rollout_corr/ppl_ratio:np.float64(1.0008906812872738) - rollout_corr/chi2_token:np.float64(0.0020458281505852938) - rollout_corr/chi2_seq:np.float64(0.6757269599474967) - actor/pg_loss:np.float64(0.0028436718275770545) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006703252311126562) - actor/ppo_kl:np.float64(0.0001225946314420412) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.76953125) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.76953125) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6015625) - self_distillation/token_reweight_w_mean:np.float64(73408.10212582699) - self_distillation/token_reweight_w_std:np.float64(1221313.2646529041) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.0004472548607737) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07176223069836851) - self_distillation/empty_target_batch:np.float64(0.23046875) - actor/grad_norm:np.float64(0.3494473174214363) - perf/mfu/actor:np.float64(0.03336222886368918) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.36438751220703) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:1 - critic/score/mean:0.6015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0025512599386274815 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0025512599386274815 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:291.1015625 - response_length/max:636.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:291.1015625 - response_length_non_aborted/max:636.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.65625 - prompt_length/max:456.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012776069343090057 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3372939508408308) - timing_s/agent_loop/generate_sequences/max:np.float64(5.064935522153974) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.084805842998321) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.064935522153974) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:338 - timing_s/agent_loop/slowest/response_length:572 - timing_s/gen:11.018641920760274 - timing_s/reward:0.05196058563888073 - timing_s/old_log_prob:2.2254901994019747 - timing_s/adv:0.4944445975124836 - timing_s/update_actor:15.122869728133082 - timing_s/step:29.00846175290644 - timing_s/stop_profile:2.9573217034339905e-05 - timing_per_token_ms/adv:0.0034628366752516605 - timing_per_token_ms/gen:0.14785757119723403 - timing_per_token_ms/update_actor:0.10591283268761001 - perf/total_num_tokens:142786 - perf/time_per_step:29.00846175290644 - perf/throughput:615.2773681014552 (TaskRunner pid=2026458) Training Progress: 96%|█████████▌| 96/100 [1:07:15<02:15, 33.85s/it] (TaskRunner pid=2026458) step:97 - global_seqlen/min:16071 - global_seqlen/max:23461 - global_seqlen/minmax_diff:7390 - global_seqlen/balanced_min:18811 - global_seqlen/balanced_max:18815 - global_seqlen/mean:18813.875 - actor/entropy:0.2445484697818756 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4984980821609497 - training/rollout_probs_diff_mean:0.005207015201449394 - training/rollout_probs_diff_std:0.013489038683474064 - training/rollout_actor_probs_pearson_corr:0.9982696175575256 - self_distillation/success_group_fraction:0.65625 - self_distillation/success_sample_fraction:0.6484375 - self_distillation/correct_sample_fraction:0.52734375 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.6484375 - rollout_corr/rollout_is_mean:0.9999282360076904 - rollout_corr/rollout_is_ratio_fraction_high:1.330795930698514e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.65397965349257e-05 - rollout_corr/rollout_is_max:3.3646888732910156 - rollout_corr/rollout_is_min:0.3885670304298401 - rollout_corr/rollout_is_std:0.04048614203929901 - rollout_corr/rollout_is_eff_sample_size:0.9983633169728505 - rollout_corr/rollout_is_seq_mean:0.9999846816062927 - rollout_corr/rollout_is_seq_std:0.002680700272321701 - rollout_corr/rollout_is_seq_max:1.0078043937683105 - rollout_corr/rollout_is_seq_min:0.9888970255851746 - rollout_corr/rollout_is_seq_max_deviation:0.01110297441482544 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3025756301358342) - rollout_corr/training_log_ppl:np.float64(0.2600360629003262) - rollout_corr/kl:np.float64(0.001345034299074399) - rollout_corr/k3_kl:np.float64(0.0010471967440253138) - rollout_corr/rollout_ppl:np.float64(1.3007347490638494) - rollout_corr/rollout_log_ppl:np.float64(0.2586910269019427) - rollout_corr/log_ppl_diff:np.float64(0.001345035998383537) - rollout_corr/log_ppl_abs_diff:np.float64(0.002600655221613124) - rollout_corr/log_ppl_diff_max:np.float64(0.013873368501663208) - rollout_corr/log_ppl_diff_min:np.float64(-0.008637338876724243) - rollout_corr/ppl_ratio:np.float64(1.0013508247211576) - rollout_corr/chi2_token:np.float64(0.001424239482730627) - rollout_corr/chi2_seq:np.float64(0.4958528857678175) - actor/pg_loss:np.float64(0.0023965955479070544) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000528412345374818) - actor/ppo_kl:np.float64(0.00046769405014401855) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.6484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.6484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.52734375) - self_distillation/token_reweight_w_mean:np.float64(76633.93915633648) - self_distillation/token_reweight_w_std:np.float64(1211350.6270655608) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.9995253349188715) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0652933964447584) - self_distillation/empty_target_batch:np.float64(0.3515625) - actor/grad_norm:np.float64(0.31651656329631805) - perf/mfu/actor:np.float64(0.03500956607334557) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.4477882385254) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:1 - critic/score/mean:0.52734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.52734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004978840239346027 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004978840239346027 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:293.52734375 - response_length/max:730.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:293.52734375 - response_length_non_aborted/max:730.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:294.40625 - prompt_length/max:900.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.985447645187378e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1220364272594452) - timing_s/agent_loop/generate_sequences/max:np.float64(5.692030295729637) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1152001094887964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.692030295729637) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:730 - timing_s/gen:11.270145887508988 - timing_s/reward:0.05703514441847801 - timing_s/old_log_prob:2.3115832544863224 - timing_s/adv:0.5203931517899036 - timing_s/update_actor:15.463589219376445 - timing_s/step:29.711128624156117 - timing_s/stop_profile:0.00012691877782344818 - timing_per_token_ms/adv:0.0034575090976068437 - timing_per_token_ms/gen:0.14998264492379848 - timing_per_token_ms/update_actor:0.10274059184628662 - perf/total_num_tokens:150511 - perf/time_per_step:29.711128624156117 - perf/throughput:633.2265340032793 (TaskRunner pid=2026458) Training Progress: 97%|█████████▋| 97/100 [1:07:45<01:37, 32.62s/it] (TaskRunner pid=2026458) step:98 - global_seqlen/min:14853 - global_seqlen/max:23186 - global_seqlen/minmax_diff:8333 - global_seqlen/balanced_min:19016 - global_seqlen/balanced_max:19021 - global_seqlen/mean:19019.875 - actor/entropy:0.23993536829948425 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2247922420501709 - training/rollout_probs_diff_mean:0.005032053682953119 - training/rollout_probs_diff_std:0.013124685734510422 - training/rollout_actor_probs_pearson_corr:0.99836266040802 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71484375 - self_distillation/correct_sample_fraction:0.58203125 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71484375 - rollout_corr/rollout_is_mean:0.9997885227203369 - rollout_corr/rollout_is_ratio_fraction_high:2.6424617317388766e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.9636925066588446e-05 - rollout_corr/rollout_is_max:2.206674814224243 - rollout_corr/rollout_is_min:0.19827817380428314 - rollout_corr/rollout_is_std:0.04013793542981148 - rollout_corr/rollout_is_eff_sample_size:0.9983908838332385 - rollout_corr/rollout_is_seq_mean:0.9997617602348328 - rollout_corr/rollout_is_seq_std:0.0025118952617049217 - rollout_corr/rollout_is_seq_max:1.0087740421295166 - rollout_corr/rollout_is_seq_min:0.9924179315567017 - rollout_corr/rollout_is_seq_max_deviation:0.008774042129516602 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2941957069560885) - rollout_corr/training_log_ppl:np.float64(0.254570292920107) - rollout_corr/kl:np.float64(0.0012180306670934726) - rollout_corr/k3_kl:np.float64(0.000970762781207668) - rollout_corr/rollout_ppl:np.float64(1.2925921510905027) - rollout_corr/rollout_log_ppl:np.float64(0.25335226202150807) - rollout_corr/log_ppl_diff:np.float64(0.001218030898598954) - rollout_corr/log_ppl_abs_diff:np.float64(0.0023932511976454407) - rollout_corr/log_ppl_diff_max:np.float64(0.008434802293777466) - rollout_corr/log_ppl_diff_min:np.float64(-0.005523428320884705) - rollout_corr/ppl_ratio:np.float64(1.0012223366647959) - rollout_corr/chi2_token:np.float64(0.0014376682229340076) - rollout_corr/chi2_seq:np.float64(0.6267413701862097) - actor/pg_loss:np.float64(0.003052193787880242) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006503687072836328) - actor/ppo_kl:np.float64(0.000135291372203028) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71484375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71484375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.58203125) - self_distillation/token_reweight_w_mean:np.float64(107416.14729232923) - self_distillation/token_reweight_w_std:np.float64(1626133.0662139303) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999873915920034) - self_distillation/token_reweight_w_clip_frac:np.float64(0.0583975906483829) - self_distillation/empty_target_batch:np.float64(0.28515625) - actor/grad_norm:np.float64(0.36374419182538986) - perf/mfu/actor:np.float64(0.03551806757350935) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.40508651733398) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:1 - critic/score/mean:0.58203125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.58203125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.001578870927914977 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.001578870927914977 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:295.65234375 - response_length/max:718.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:295.65234375 - response_length_non_aborted/max:718.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:298.71875 - prompt_length/max:631.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011442229151725769 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4412503708153963) - timing_s/agent_loop/generate_sequences/max:np.float64(5.591642700135708) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.1361403591217822) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.591642700135708) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:718 - timing_s/gen:11.411643352359533 - timing_s/reward:0.05464274063706398 - timing_s/old_log_prob:2.2537942565977573 - timing_s/adv:0.5091272983700037 - timing_s/update_actor:15.217187775298953 - timing_s/step:29.54008462652564 - timing_s/stop_profile:0.00011652149260044098 - timing_per_token_ms/adv:0.003346021585118223 - timing_per_token_ms/gen:0.1507741534525022 - timing_per_token_ms/update_actor:0.1000084633528017 - perf/total_num_tokens:152159 - perf/time_per_step:29.54008462652564 - perf/throughput:643.8666388559031 (TaskRunner pid=2026458) Training Progress: 98%|█████████▊| 98/100 [1:08:14<01:03, 31.71s/it] (TaskRunner pid=2026458) step:99 - global_seqlen/min:16270 - global_seqlen/max:23777 - global_seqlen/minmax_diff:7507 - global_seqlen/balanced_min:18894 - global_seqlen/balanced_max:18898 - global_seqlen/mean:18896.125 - actor/entropy:0.23954525589942932 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5315115451812744 - training/rollout_probs_diff_mean:0.005148096010088921 - training/rollout_probs_diff_std:0.013385861180722713 - training/rollout_actor_probs_pearson_corr:0.9983036518096924 - self_distillation/success_group_fraction:0.8125 - self_distillation/success_sample_fraction:0.80859375 - self_distillation/correct_sample_fraction:0.640625 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.80859375 - rollout_corr/rollout_is_mean:1.00002920627594 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.40447464218596e-05 - rollout_corr/rollout_is_max:1.9394363164901733 - rollout_corr/rollout_is_min:0.2951362729072571 - rollout_corr/rollout_is_std:0.04090065881609917 - rollout_corr/rollout_is_eff_sample_size:0.9983299298399158 - rollout_corr/rollout_is_seq_mean:1.0000901222229004 - rollout_corr/rollout_is_seq_std:0.0026464853435754776 - rollout_corr/rollout_is_seq_max:1.00861656665802 - rollout_corr/rollout_is_seq_min:0.9930309057235718 - rollout_corr/rollout_is_seq_max_deviation:0.00861656665802002 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2940249065868556) - rollout_corr/training_log_ppl:np.float64(0.25375668989727274) - rollout_corr/kl:np.float64(0.0008266884788987738) - rollout_corr/k3_kl:np.float64(0.0009614558624662095) - rollout_corr/rollout_ppl:np.float64(1.2929239594377577) - rollout_corr/rollout_log_ppl:np.float64(0.2529299992165761) - rollout_corr/log_ppl_diff:np.float64(0.000826690680696629) - rollout_corr/log_ppl_abs_diff:np.float64(0.0025123875675490126) - rollout_corr/log_ppl_diff_max:np.float64(0.009278476238250732) - rollout_corr/log_ppl_diff_min:np.float64(-0.007872521877288818) - rollout_corr/ppl_ratio:np.float64(1.000831762328744) - rollout_corr/chi2_token:np.float64(0.002216932363808155) - rollout_corr/chi2_seq:np.float64(0.6580137938726693) - actor/pg_loss:np.float64(0.004063432686962187) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007904909580247477) - actor/ppo_kl:np.float64(4.468652047506794e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.80859375) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.80859375) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.640625) - self_distillation/token_reweight_w_mean:np.float64(131557.28193879593) - self_distillation/token_reweight_w_std:np.float64(2127779.07550747) - self_distillation/token_reweight_w_clipped_mean:np.float64(0.999837173614651) - self_distillation/token_reweight_w_clip_frac:np.float64(0.07958601297286805) - self_distillation/empty_target_batch:np.float64(0.19140625) - actor/grad_norm:np.float64(0.4794868528842926) - perf/mfu/actor:np.float64(0.035181320287522105) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.34246444702148) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:1 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003239069599658251 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003239069599658251 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:278.12890625 - response_length/max:681.0 - response_length/min:101.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:278.12890625 - response_length_non_aborted/max:681.0 - response_length_non_aborted/min:101.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:312.375 - prompt_length/max:836.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017566047608852386 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2292504087090492) - timing_s/agent_loop/generate_sequences/max:np.float64(5.23788345977664) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.0256088213573094) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.23788345977664) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:203 - timing_s/agent_loop/slowest/response_length:681 - timing_s/gen:11.295774277299643 - timing_s/reward:0.05612022429704666 - timing_s/old_log_prob:2.242926351726055 - timing_s/adv:0.5197359751909971 - timing_s/update_actor:15.403783183544874 - timing_s/step:29.607021318748593 - timing_s/stop_profile:0.00011566095054149628 - timing_per_token_ms/adv:0.0034381121472722392 - timing_per_token_ms/gen:0.1586462869524254 - timing_per_token_ms/update_actor:0.101897764644503 - perf/total_num_tokens:151169 - perf/time_per_step:29.607021318748593 - perf/throughput:638.2312086232755 (TaskRunner pid=2026458) Training Progress: 99%|█████████▉| 99/100 [1:08:44<00:31, 31.09s/it] (TaskRunner pid=2026458) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2026458) validation generation end (TaskRunner pid=2026458) [prompt] system (TaskRunner pid=2026458) (TaskRunner pid=2026458) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) ... (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) user (TaskRunner pid=2026458) Which of the following molecules is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" ? (TaskRunner pid=2026458) (TaskRunner pid=2026458) A: O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1 (TaskRunner pid=2026458) B: O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1 (TaskRunner pid=2026458) C: O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1 (TaskRunner pid=2026458) D: O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F (TaskRunner pid=2026458) Please reason step by step. (TaskRunner pid=2026458) assistant (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) [response] (TaskRunner pid=2026458) The question asks which of the given options is the only correct reactant used in the synthesis of the product "Nc1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1". The product contains a structure with a chlorine atom, a trifluoromethyl group (C(F)(F)F), and a specific aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option A: "O=C1c2ccccc2C(=O)N1c1ccn(Cc2ccc(Cl)cc2C(F)(F)F)n1" contains the correct structure with a chlorine atom, a trifluoromethyl group, and the necessary aromatic ring, matching the product structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option B: "O=C1c2ccccc2C(Cl)N1c1ccc2ccc(Cl)nc2n1" contains a chlorine atom but lacks the correct trifluoromethyl group and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option C: "O=C1c2ccccc2C(=O)N1Cc1ccc(Oc2ccc(Cl)c(C(F)(F)F)c2)cc1" contains a trifluoromethyl group and a chlorine atom but has an incorrect aromatic ring structure and additional functional groups not present in the product. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Option D: "O=C1c2ccccc2C(=O)N1CC=C(Cc1ccccc1)C(F)(F)F" contains a trifluoromethyl group but lacks the correct chlorine atom and the correct aromatic ring structure. (TaskRunner pid=2026458) (TaskRunner pid=2026458) Therefore, the only option that matches the product structure is Option A. (TaskRunner pid=2026458) (TaskRunner pid=2026458) (TaskRunner pid=2026458) A (TaskRunner pid=2026458) (TaskRunner pid=2026458) [ground_truth] A (TaskRunner pid=2026458) [score] 1.0 (TaskRunner pid=2026458) [acc] 1.0 (TaskRunner pid=2026458) [pred] A (TaskRunner pid=2026458) [incorrect_format] 1 (TaskRunner pid=2026458) [feedback] (TaskRunner pid=2026458) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2026868) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/chemistry/qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2026458) step:100 - global_seqlen/min:17070 - global_seqlen/max:22010 - global_seqlen/minmax_diff:4940 - global_seqlen/balanced_min:19621 - global_seqlen/balanced_max:19622 - global_seqlen/mean:19621.75 - actor/entropy:0.21838146448135376 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30163663625717163 - training/rollout_probs_diff_mean:0.004920980893075466 - training/rollout_probs_diff_std:0.013346283696591854 - training/rollout_actor_probs_pearson_corr:0.9981935024261475 - self_distillation/success_group_fraction:0.71875 - self_distillation/success_sample_fraction:0.71875 - self_distillation/correct_sample_fraction:0.6171875 - self_distillation/feedback_available_fraction:0.0 - self_distillation/feedback_used_fraction:0.0 - self_distillation/reprompt_sample_fraction:0.71875 - rollout_corr/rollout_is_mean:1.0000070333480835 - rollout_corr/rollout_is_ratio_fraction_high:1.2321644135226961e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.16082179476507e-05 - rollout_corr/rollout_is_max:2.1037662029266357 - rollout_corr/rollout_is_min:0.39213550090789795 - rollout_corr/rollout_is_std:0.04028838127851486 - rollout_corr/rollout_is_eff_sample_size:0.9983795954951445 - rollout_corr/rollout_is_seq_mean:1.0000463724136353 - rollout_corr/rollout_is_seq_std:0.002407139865681529 - rollout_corr/rollout_is_seq_max:1.0090001821517944 - rollout_corr/rollout_is_seq_min:0.9934689998626709 - rollout_corr/rollout_is_seq_max_deviation:0.009000182151794434 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2685748748481274) - rollout_corr/training_log_ppl:np.float64(0.23310650100756902) - rollout_corr/kl:np.float64(0.0009287315268036167) - rollout_corr/k3_kl:np.float64(0.001084987000297133) - rollout_corr/rollout_ppl:np.float64(1.2673596912063658) - rollout_corr/rollout_log_ppl:np.float64(0.2321777678298531) - rollout_corr/log_ppl_diff:np.float64(0.0009287331777159125) - rollout_corr/log_ppl_abs_diff:np.float64(0.002280793822137639) - rollout_corr/log_ppl_diff_max:np.float64(0.015468984842300415) - rollout_corr/log_ppl_diff_min:np.float64(-0.010868668556213379) - rollout_corr/ppl_ratio:np.float64(1.0009334608912468) - rollout_corr/chi2_token:np.float64(0.002497109118849039) - rollout_corr/chi2_seq:np.float64(1.0585334873758256) - actor/pg_loss:np.float64(0.0023272527614608407) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004233177678543143) - actor/ppo_kl:np.float64(0.00010989303795483352) - actor/pg_clipfrac_lower:np.float64(0.0) - self_distillation/token_reweight_lambda:np.float64(0.5) - self_distillation/token_reweight_eps_w:np.float64(0.20000000000000004) - self_distillation/token_reweight_active_token_fraction:np.float64(0.71875) - self_distillation/token_reweight_context_sample_fraction:np.float64(0.71875) - self_distillation/token_reweight_correct_sample_fraction:np.float64(0.6171875) - self_distillation/token_reweight_w_mean:np.float64(73094.00544022233) - self_distillation/token_reweight_w_std:np.float64(1188161.9467263469) - self_distillation/token_reweight_w_clipped_mean:np.float64(1.000056762015447) - self_distillation/token_reweight_w_clip_frac:np.float64(0.03758277394808829) - self_distillation/empty_target_batch:np.float64(0.28125) - actor/grad_norm:np.float64(0.25844827108085155) - perf/mfu/actor:np.float64(0.03632957727570209) - perf/max_memory_allocated_gb:np.float64(124.80831956863403) - perf/max_memory_reserved_gb:np.float64(131.27734375) - perf/cpu_memory_used_gb:np.float64(146.29161834716797) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/reward/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.05779624954713381) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8013714285714286) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.03927314998986781) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7074904761904762) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.03834834240388913) - val-aux/sciknoweval/score/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/score/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/score/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/score/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/score/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.05779624954713381) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8013714285714286) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.03927314998986781) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7074904761904762) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.03834834240388913) - val-core/sciknoweval/acc/mean@16:np.float64(0.6895833333333333) - val-aux/sciknoweval/acc/std@16:np.float64(0.10973085235155812) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.733895238095238) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.08320072408312382) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6464904761904762) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.10098054289944847) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6907285714285715) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.10956288078402857) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7644238095238095) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.05746985351073585) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.6048666666666667) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.08537783027231041) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6991761904761905) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.08403267534041585) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7858285714285714) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.038120394142420314) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.5691952380952381) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.06319778055664713) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.7047571428571429) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.05779624954713381) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8013714285714286) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02391792727193744) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.5435285714285715) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.03927314998986781) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7074904761904762) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.03834834240388913) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.999702380952381) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0011526736149426837) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9999904761904761) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0002127457895589398) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9994476190476191) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0015248805759714617) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9997) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.0011569664768572288) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9988761904761905) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0020220104047945714) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9999476190476191) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.0004966783087710074) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9980238095238095) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.002346295386522406) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9967095238095238) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0022003864842959084) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:1 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0011474530911073089 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0011474530911073089 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:317.0234375 - response_length/max:616.0 - response_length/min:122.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:317.0234375 - response_length_non_aborted/max:616.0 - response_length_non_aborted/min:122.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:296.15625 - prompt_length/max:625.0 - prompt_length/min:165.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00018987245857715607 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5517499037086964) - timing_s/agent_loop/generate_sequences/max:np.float64(5.5460103284567595) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.556732240387646) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(5.5460103284567595) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:232 - timing_s/agent_loop/slowest/response_length:602 - timing_s/gen:11.436559945344925 - timing_s/reward:0.05615238472819328 - timing_s/old_log_prob:2.225490480661392 - timing_s/adv:0.4910304043442011 - timing_s/update_actor:15.488128297030926 - timing_s/step:29.756166722625494 - timing_s/testing:108.75369282439351 - timing_s/save_checkpoint:7.179236024618149 - timing_s/stop_profile:0.00012516416609287262 - timing_per_token_ms/adv:0.003128100222611395 - timing_per_token_ms/gen:0.14091722252082264 - timing_per_token_ms/update_actor:0.0986668384384097 - perf/total_num_tokens:156974 - perf/time_per_step:29.756166722625494 - perf/throughput:659.4179345379304 (TaskRunner pid=2026458) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/reward/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/reward/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-aux/sciknoweval/reward/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-aux/sciknoweval/reward/maj@16/std': " (TaskRunner pid=2026458) "np.float64(0.03834834240388913), 'val-aux/sciknoweval/score/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/score/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/score/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/score/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/score/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/score/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/score/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/score/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/score/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/score/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/score/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/score/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/score/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/score/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/score/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/score/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/score/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/score/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/score/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/score/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-aux/sciknoweval/score/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-aux/sciknoweval/score/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/score/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/score/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-aux/sciknoweval/score/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-aux/sciknoweval/score/maj@16/std': " (TaskRunner pid=2026458) "np.float64(0.03834834240388913), 'val-core/sciknoweval/acc/mean@16': " (TaskRunner pid=2026458) "np.float64(0.6895833333333333), 'val-aux/sciknoweval/acc/std@16': " (TaskRunner pid=2026458) "np.float64(0.10973085235155812), 'val-aux/sciknoweval/acc/best@2/mean': " (TaskRunner pid=2026458) "np.float64(0.733895238095238), 'val-aux/sciknoweval/acc/best@2/std': " (TaskRunner pid=2026458) "np.float64(0.08320072408312382), 'val-aux/sciknoweval/acc/worst@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6464904761904762), 'val-aux/sciknoweval/acc/worst@2/std': " (TaskRunner pid=2026458) "np.float64(0.10098054289944847), 'val-aux/sciknoweval/acc/maj@2/mean': " (TaskRunner pid=2026458) "np.float64(0.6907285714285715), 'val-aux/sciknoweval/acc/maj@2/std': " (TaskRunner pid=2026458) "np.float64(0.10956288078402857), 'val-aux/sciknoweval/acc/best@4/mean': " (TaskRunner pid=2026458) "np.float64(0.7644238095238095), 'val-aux/sciknoweval/acc/best@4/std': " (TaskRunner pid=2026458) "np.float64(0.05746985351073585), 'val-aux/sciknoweval/acc/worst@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6048666666666667), 'val-aux/sciknoweval/acc/worst@4/std': " (TaskRunner pid=2026458) "np.float64(0.08537783027231041), 'val-aux/sciknoweval/acc/maj@4/mean': " (TaskRunner pid=2026458) "np.float64(0.6991761904761905), 'val-aux/sciknoweval/acc/maj@4/std': " (TaskRunner pid=2026458) "np.float64(0.08403267534041585), 'val-aux/sciknoweval/acc/best@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7858285714285714), 'val-aux/sciknoweval/acc/best@8/std': " (TaskRunner pid=2026458) "np.float64(0.038120394142420314), 'val-aux/sciknoweval/acc/worst@8/mean': " (TaskRunner pid=2026458) "np.float64(0.5691952380952381), 'val-aux/sciknoweval/acc/worst@8/std': " (TaskRunner pid=2026458) "np.float64(0.06319778055664713), 'val-aux/sciknoweval/acc/maj@8/mean': " (TaskRunner pid=2026458) "np.float64(0.7047571428571429), 'val-aux/sciknoweval/acc/maj@8/std': " (TaskRunner pid=2026458) "np.float64(0.05779624954713381), 'val-core/sciknoweval/acc/best@16/mean': " (TaskRunner pid=2026458) "np.float64(0.8013714285714286), 'val-core/sciknoweval/acc/best@16/std': " (TaskRunner pid=2026458) "np.float64(0.02391792727193744), 'val-aux/sciknoweval/acc/worst@16/mean': " (TaskRunner pid=2026458) "np.float64(0.5435285714285715), 'val-aux/sciknoweval/acc/worst@16/std': " (TaskRunner pid=2026458) "np.float64(0.03927314998986781), 'val-core/sciknoweval/acc/maj@16/mean': " (TaskRunner pid=2026458) "np.float64(0.7074904761904762), 'val-core/sciknoweval/acc/maj@16/std': " (TaskRunner pid=2026458) 'np.float64(0.03834834240388913), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/mean@16': " (TaskRunner pid=2026458) 'np.float64(0.999702380952381), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2026458) 'np.float64(0.0011526736149426837), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@2/mean': " (TaskRunner pid=2026458) 'np.float64(0.9999904761904761), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0002127457895589398), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2026458) 'np.float64(0.9994476190476191), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0015248805759714617), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9997), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2026458) 'np.float64(0.0011569664768572288), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@4/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@4/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': " (TaskRunner pid=2026458) 'np.float64(0.9988761904761905), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2026458) 'np.float64(0.0020220104047945714), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2026458) 'np.float64(0.9999476190476191), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2026458) 'np.float64(0.0004966783087710074), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': " (TaskRunner pid=2026458) 'np.float64(0.9980238095238095), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2026458) 'np.float64(0.002346295386522406), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@8/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2026458) 'np.float64(0.9967095238095238), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2026458) 'np.float64(0.0022003864842959084), ' (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': np.float64(1.0), " (TaskRunner pid=2026458) "'val-aux/sciknoweval/incorrect_format/maj@16/std': np.float64(0.0), " (TaskRunner pid=2026458) "'val-aux/num_turns/min': np.int32(2), 'val-aux/num_turns/max': np.int32(2), " (TaskRunner pid=2026458) "'val-aux/num_turns/mean': np.float64(2.0)}") (TaskRunner pid=2026458) Training Progress: 100%|██████████| 100/100 [1:11:10<00:00, 65.49s/it] Training Progress: 100%|██████████| 100/100 [1:11:10<00:00, 42.70s/it] (TaskRunner pid=2026458) wandb: updating run metadata (TaskRunner pid=2026458) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2026458) wandb: uploading output.log (TaskRunner pid=2026458) wandb: uploading history steps 98-99, summary, console lines 651-774 (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: Run history: (TaskRunner pid=2026458) wandb: actor/entropy ▂▁▂▅▄█▅▅▃▆▆▆▇█▆▅▆▆▄▆▃▂▄▅▄▃▃▄▃▄▃▃▄▂▂▂▂▂▃▂ (TaskRunner pid=2026458) wandb: actor/grad_norm ▆▄▆▆▆▄▆▅▄▅▆▇▅▄▄▄▃▄▃▃▄▃█▃▄▄▃▁▂▃▃▂▂▃▃▁▃▃▂▂ (TaskRunner pid=2026458) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2026458) wandb: actor/lr ▁▃▄▅▅▇██████████████████████████████████ (TaskRunner pid=2026458) wandb: actor/pg_clipfrac ▁▅▆▆▅▅▄▅▃█▃▆▄▄▆▃▃▆▃▃▃▅▃▇▆▄▃▄▄▄▅▃▃▄▅▂▄▂▅▂ (TaskRunner pid=2026458) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▅▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃▁▁▁▁▁▁▁ (TaskRunner pid=2026458) wandb: actor/pg_loss █▅▅▆▆▄▇▅▄▄▃▃▃▄▁▂▃▂▄▃▃▃▃▂▃▂▃▂▂▂▃▃▃▄▄▁▂▂▂▂ (TaskRunner pid=2026458) wandb: actor/ppo_kl ▃▄▁▃▆▂▅▃▄▂▁▃▃█▆▆▄▄▅▅▃▅▆▂▆▆▆▂█▄▃▄▆▄▅▆█▄▇▄ (TaskRunner pid=2026458) wandb: critic/advantages/max █████████▃██▆████████████████▆██▆▆▆▁████ (TaskRunner pid=2026458) wandb: critic/advantages/mean ▃█▅▄▃▂▃▄▁▂▃▄▄▁▄▃▄▂▃▅▄▄▃▄▃▂▄▃▃▄▄▂▄▁▄▃▃▄▂▃ (TaskRunner pid=2026458) wandb: +220 ... (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: Run summary: (TaskRunner pid=2026458) wandb: actor/entropy 0.21838 (TaskRunner pid=2026458) wandb: actor/grad_norm 0.25845 (TaskRunner pid=2026458) wandb: actor/kl_loss 0 (TaskRunner pid=2026458) wandb: actor/lr 0.0 (TaskRunner pid=2026458) wandb: actor/pg_clipfrac 0.00042 (TaskRunner pid=2026458) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2026458) wandb: actor/pg_loss 0.00233 (TaskRunner pid=2026458) wandb: actor/ppo_kl 0.00011 (TaskRunner pid=2026458) wandb: critic/advantages/max 0.75 (TaskRunner pid=2026458) wandb: critic/advantages/mean -0.00115 (TaskRunner pid=2026458) wandb: +220 ... (TaskRunner pid=2026458) wandb: (TaskRunner pid=2026458) wandb: 🚀 View run qwen3gen-chemistry-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/bz6p2yxy (TaskRunner pid=2026458) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2026458) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2026458) wandb: Find logs at: ./wandb/run-20260702_062054-bz6p2yxy/logs (TaskRunner pid=2026458) Exception ignored in atexit callback: .teardown_atexit at 0x7a92e933fb00> (TaskRunner pid=2026458) Traceback (most recent call last): (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 65, in teardown_atexit (TaskRunner pid=2026458) conn.teardown(hooks.exit_code) (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 299, in teardown (TaskRunner pid=2026458) self._asyncer.run(publish_teardown_and_close) (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 136, in run (TaskRunner pid=2026458) return future.result() (TaskRunner pid=2026458) ^^^^^^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 456, in result (TaskRunner pid=2026458) return self.__get_result() (TaskRunner pid=2026458) ^^^^^^^^^^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result (TaskRunner pid=2026458) raise self._exception (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/asyncio_manager.py", line 219, in _wrap (TaskRunner pid=2026458) return await fn() (TaskRunner pid=2026458) ^^^^^^^^^^ (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_connection.py", line 297, in publish_teardown_and_close (TaskRunner pid=2026458) await self._client.close() (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/wandb/sdk/lib/service/service_client.py", line 69, in close (TaskRunner pid=2026458) await self._writer.wait_closed() (TaskRunner pid=2026458) File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/asyncio/streams.py", line 364, in wait_closed (TaskRunner pid=2026458) await self._protocol._get_close_waiter(self) (TaskRunner pid=2026458) BrokenPipeError: [Errno 32] Broken pipe main_ppo exit code: 0 [2026-07-02 07:32:12] Finished chemistry rlsd_tr [2026-07-02 07:32:12] Starting physics grpo Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: grpo Config: baseline_grpo Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_grpo_Qwen3_4B 2026-07-02 07:32:14,409 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 Config: baseline_grpo Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-GRPO-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_grpo_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.actor.policy_loss.loss_mode=vanilla actor_rollout_ref.actor.kl_loss_coef=0.0 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 07:32:24,172 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2049544) TaskRunner hostname: mole-workspace-rw, PID: 2049544 (TaskRunner pid=2049544) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2049544) 'calculate_entropy': False, (TaskRunner pid=2049544) 'calculate_sum_pi_squared': False, (TaskRunner pid=2049544) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2049544) 'async_save': False, (TaskRunner pid=2049544) 'load_contents': ['model', (TaskRunner pid=2049544) 'optimizer', (TaskRunner pid=2049544) 'extra'], (TaskRunner pid=2049544) 'save_contents': ['model', (TaskRunner pid=2049544) 'optimizer', (TaskRunner pid=2049544) 'extra']}, (TaskRunner pid=2049544) 'clip_ratio': 0.2, (TaskRunner pid=2049544) 'clip_ratio_c': 3.0, (TaskRunner pid=2049544) 'clip_ratio_high': 0.28, (TaskRunner pid=2049544) 'clip_ratio_low': 0.2, (TaskRunner pid=2049544) 'data_loader_seed': 42, (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_coeff': 0, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'freeze_vision_tower': False, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': False, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'grad_clip': 1.0, (TaskRunner pid=2049544) 'kl_loss_coef': 0.0, (TaskRunner pid=2049544) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2049544) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2049544) 'loss_scale_factor': None, (TaskRunner pid=2049544) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2049544) 'betas': [0.9, 0.999], (TaskRunner pid=2049544) 'clip_grad': 1.0, (TaskRunner pid=2049544) 'lr': 1e-06, (TaskRunner pid=2049544) 'lr_scheduler_type': 'constant', (TaskRunner pid=2049544) 'lr_warmup_steps': 10, (TaskRunner pid=2049544) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2049544) 'min_lr_ratio': 0.0, (TaskRunner pid=2049544) 'num_cycles': 0.5, (TaskRunner pid=2049544) 'optimizer': 'AdamW', (TaskRunner pid=2049544) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2049544) 'override_optimizer_config': None, (TaskRunner pid=2049544) 'total_training_steps': -1, (TaskRunner pid=2049544) 'warmup_style': None, (TaskRunner pid=2049544) 'weight_decay': 0.01}, (TaskRunner pid=2049544) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2049544) 'clip_cov_lb': 1.0, (TaskRunner pid=2049544) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2049544) 'clip_cov_ub': 5.0, (TaskRunner pid=2049544) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2049544) 'loss_mode': 'vanilla', (TaskRunner pid=2049544) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2049544) 'ppo_epochs': 1, (TaskRunner pid=2049544) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'ppo_micro_batch_size': None, (TaskRunner pid=2049544) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'ppo_mini_batch_size': 8, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2049544) 'mode': 'disabled', (TaskRunner pid=2049544) 'record_file': None, (TaskRunner pid=2049544) 'replay_file': None}, (TaskRunner pid=2049544) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2049544) 'alpha': 0.5, (TaskRunner pid=2049544) 'distillation_add_tail': True, (TaskRunner pid=2049544) 'distillation_topk': 100, (TaskRunner pid=2049544) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2049544) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2049544) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'loss_weight': 0.0, (TaskRunner pid=2049544) 'mask': 'all'}, (TaskRunner pid=2049544) 'feedback_template': '\n' (TaskRunner pid=2049544) 'The ' (TaskRunner pid=2049544) 'following ' (TaskRunner pid=2049544) 'is ' (TaskRunner pid=2049544) 'feedback ' (TaskRunner pid=2049544) 'from ' (TaskRunner pid=2049544) 'your ' (TaskRunner pid=2049544) 'unsuccessful ' (TaskRunner pid=2049544) 'earlier ' (TaskRunner pid=2049544) 'attempt:\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) '{feedback_raw}', (TaskRunner pid=2049544) 'full_logit_distillation': True, (TaskRunner pid=2049544) 'include_environment_feedback': True, (TaskRunner pid=2049544) 'is_clip': 2, (TaskRunner pid=2049544) 'max_reprompt_len': 22528, (TaskRunner pid=2049544) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2049544) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) 'Correctly ' (TaskRunner pid=2049544) 'solve ' (TaskRunner pid=2049544) 'the ' (TaskRunner pid=2049544) 'original ' (TaskRunner pid=2049544) 'question.', (TaskRunner pid=2049544) 'reprompt_truncation': 'right', (TaskRunner pid=2049544) 'solution_template': '\n' (TaskRunner pid=2049544) 'Correct ' (TaskRunner pid=2049544) 'solution:\n' (TaskRunner pid=2049544) '\n' (TaskRunner pid=2049544) '{successful_previous_attempt}', (TaskRunner pid=2049544) 'srpo_dynamic_weighting': False, (TaskRunner pid=2049544) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2049544) 'success_reward_threshold': 0.5, (TaskRunner pid=2049544) 'teacher_regularization': 'ema', (TaskRunner pid=2049544) 'teacher_update_rate': 0.0, (TaskRunner pid=2049544) 'token_reweight_decay_steps': None, (TaskRunner pid=2049544) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2049544) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2049544) 'shuffle': False, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2049544) 'tau_neg': 1.05, (TaskRunner pid=2049544) 'tau_pos': 1.0, (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_kl_loss': False, (TaskRunner pid=2049544) 'use_prefix_grouper': False, (TaskRunner pid=2049544) 'use_remove_padding': True, (TaskRunner pid=2049544) 'use_torch_compile': True}, (TaskRunner pid=2049544) 'hybrid_engine': True, (TaskRunner pid=2049544) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2049544) 'custom_chat_template': None, (TaskRunner pid=2049544) 'enable_activation_offload': False, (TaskRunner pid=2049544) 'enable_gradient_checkpointing': True, (TaskRunner pid=2049544) 'exclude_modules': None, (TaskRunner pid=2049544) 'external_lib': None, (TaskRunner pid=2049544) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2049544) 'hf_config_path': None, (TaskRunner pid=2049544) 'lora_adapter_path': None, (TaskRunner pid=2049544) 'lora_alpha': 16, (TaskRunner pid=2049544) 'lora_rank': 0, (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'target_modules': 'all-linear', (TaskRunner pid=2049544) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2049544) 'num_shards': 4}, (TaskRunner pid=2049544) 'tokenizer_path': None, (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_liger': False, (TaskRunner pid=2049544) 'use_remove_padding': True, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'nccl_timeout': 600, (TaskRunner pid=2049544) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': True, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'log_prob_micro_batch_size': None, (TaskRunner pid=2049544) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2049544) 'mode': 'disabled', (TaskRunner pid=2049544) 'record_file': None, (TaskRunner pid=2049544) 'replay_file': None}, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_torch_compile': True}, (TaskRunner pid=2049544) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2049544) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2049544) 'agent_loop_config_path': None, (TaskRunner pid=2049544) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2049544) 'name': None, (TaskRunner pid=2049544) 'path': None}, (TaskRunner pid=2049544) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2049544) 'num_workers': 8}, (TaskRunner pid=2049544) 'calculate_log_probs': True, (TaskRunner pid=2049544) 'cudagraph_capture_sizes': None, (TaskRunner pid=2049544) 'data_parallel_size': 1, (TaskRunner pid=2049544) 'disable_log_stats': True, (TaskRunner pid=2049544) 'do_sample': True, (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'enable_chunked_prefill': True, (TaskRunner pid=2049544) 'enable_prefix_caching': True, (TaskRunner pid=2049544) 'enable_rollout_routing_replay': False, (TaskRunner pid=2049544) 'enforce_eager': False, (TaskRunner pid=2049544) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2049544) 'expert_parallel_size': 1, (TaskRunner pid=2049544) 'free_cache_engine': True, (TaskRunner pid=2049544) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2049544) 'ignore_eos': False, (TaskRunner pid=2049544) 'layered_summon': False, (TaskRunner pid=2049544) 'load_format': 'dummy', (TaskRunner pid=2049544) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2049544) 'log_prob_micro_batch_size': None, (TaskRunner pid=2049544) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2049544) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2049544) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2049544) 'max_model_len': 10240, (TaskRunner pid=2049544) 'max_num_batched_tokens': 10240, (TaskRunner pid=2049544) 'max_num_seqs': 1024, (TaskRunner pid=2049544) 'mode': 'async', (TaskRunner pid=2049544) 'multi_stage_wake_up': False, (TaskRunner pid=2049544) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'format': 'hermes', (TaskRunner pid=2049544) 'interaction_config_path': None, (TaskRunner pid=2049544) 'max_assistant_turns': None, (TaskRunner pid=2049544) 'max_parallel_calls': 1, (TaskRunner pid=2049544) 'max_tool_response_length': 256, (TaskRunner pid=2049544) 'max_user_turns': None, (TaskRunner pid=2049544) 'num_repeat_rollouts': None, (TaskRunner pid=2049544) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2049544) 'tool_config_path': None, (TaskRunner pid=2049544) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2049544) 'use_inference_chat_template': False}, (TaskRunner pid=2049544) 'n': 8, (TaskRunner pid=2049544) 'name': 'vllm', (TaskRunner pid=2049544) 'over_sample_rate': 0, (TaskRunner pid=2049544) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2049544) 'port': 9090, (TaskRunner pid=2049544) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2049544) 'prompt_length': 2048, (TaskRunner pid=2049544) 'quantization': None, (TaskRunner pid=2049544) 'quantization_config_file': None, (TaskRunner pid=2049544) 'response_length': 8192, (TaskRunner pid=2049544) 'scheduling_policy': 'fcfs', (TaskRunner pid=2049544) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2049544) 'skip_rollout': False, (TaskRunner pid=2049544) 'skip_tokenizer_init': True, (TaskRunner pid=2049544) 'temperature': 1.0, (TaskRunner pid=2049544) 'tensor_model_parallel_size': 2, (TaskRunner pid=2049544) 'top_k': -1, (TaskRunner pid=2049544) 'top_p': 1.0, (TaskRunner pid=2049544) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2049544) 'backend': None, (TaskRunner pid=2049544) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2049544) 'token2text': False}, (TaskRunner pid=2049544) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2049544) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2049544) 'do_sample': True, (TaskRunner pid=2049544) 'n': 16, (TaskRunner pid=2049544) 'temperature': 0.6, (TaskRunner pid=2049544) 'top_k': -1, (TaskRunner pid=2049544) 'top_p': 0.95}}}, (TaskRunner pid=2049544) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2049544) 'adv_estimator': 'grpo', (TaskRunner pid=2049544) 'gamma': 1.0, (TaskRunner pid=2049544) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2049544) 'horizon': 10000, (TaskRunner pid=2049544) 'kl_coef': 0.001, (TaskRunner pid=2049544) 'target_kl': 0.1, (TaskRunner pid=2049544) 'type': 'fixed'}, (TaskRunner pid=2049544) 'kl_penalty': 'kl', (TaskRunner pid=2049544) 'lam': 1.0, (TaskRunner pid=2049544) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2049544) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2049544) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2049544) 'loss_type': 'ppo_clip', (TaskRunner pid=2049544) 'rollout_is': 'token', (TaskRunner pid=2049544) 'rollout_is_batch_normalize': False, (TaskRunner pid=2049544) 'rollout_is_threshold': 2.0, (TaskRunner pid=2049544) 'rollout_rs': None, (TaskRunner pid=2049544) 'rollout_rs_threshold': None}, (TaskRunner pid=2049544) 'use_kl_in_reward': False, (TaskRunner pid=2049544) 'use_pf_ppo': False}, (TaskRunner pid=2049544) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2049544) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2049544) 'async_save': False, (TaskRunner pid=2049544) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2049544) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2049544) 'cliprange_value': 0.5, (TaskRunner pid=2049544) 'data_loader_seed': 42, (TaskRunner pid=2049544) 'enable': None, (TaskRunner pid=2049544) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'forward_micro_batch_size': None, (TaskRunner pid=2049544) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'grad_clip': 1.0, (TaskRunner pid=2049544) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2049544) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2049544) 'enable_activation_offload': False, (TaskRunner pid=2049544) 'enable_gradient_checkpointing': True, (TaskRunner pid=2049544) 'external_lib': None, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'entropy_checkpointing': False, (TaskRunner pid=2049544) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2049544) 'forward_only': False, (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'full_determinism': False, (TaskRunner pid=2049544) 'model_dtype': 'fp32', (TaskRunner pid=2049544) 'offload_policy': False, (TaskRunner pid=2049544) 'optimizer_offload': False, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'seed': 42, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_orig_params': False, (TaskRunner pid=2049544) 'use_torch_compile': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'lora_alpha': 16, (TaskRunner pid=2049544) 'lora_rank': 0, (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2049544) 'target_modules': 'all-linear', (TaskRunner pid=2049544) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2049544) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_remove_padding': False, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2049544) 'betas': [0.9, 0.999], (TaskRunner pid=2049544) 'clip_grad': 1.0, (TaskRunner pid=2049544) 'lr': 1e-05, (TaskRunner pid=2049544) 'lr_scheduler_type': 'constant', (TaskRunner pid=2049544) 'lr_warmup_steps': -1, (TaskRunner pid=2049544) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2049544) 'min_lr_ratio': 0.0, (TaskRunner pid=2049544) 'num_cycles': 0.5, (TaskRunner pid=2049544) 'optimizer': 'AdamW', (TaskRunner pid=2049544) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2049544) 'override_optimizer_config': None, (TaskRunner pid=2049544) 'total_training_steps': -1, (TaskRunner pid=2049544) 'warmup_style': None, (TaskRunner pid=2049544) 'weight_decay': 0.01}, (TaskRunner pid=2049544) 'ppo_epochs': 1, (TaskRunner pid=2049544) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'ppo_micro_batch_size': None, (TaskRunner pid=2049544) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'ppo_mini_batch_size': 8, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'rollout_n': 8, (TaskRunner pid=2049544) 'shuffle': False, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False}, (TaskRunner pid=2049544) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2049544) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2049544) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2049544) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2049544) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2049544) 'dataloader_num_workers': 8, (TaskRunner pid=2049544) 'filter_overlong_prompts': True, (TaskRunner pid=2049544) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2049544) 'image_key': 'images', (TaskRunner pid=2049544) 'image_patch_size': 14, (TaskRunner pid=2049544) 'max_prompt_length': 2048, (TaskRunner pid=2049544) 'max_response_length': 8192, (TaskRunner pid=2049544) 'prompt_key': 'prompt', (TaskRunner pid=2049544) 'return_full_prompt': False, (TaskRunner pid=2049544) 'return_multi_modal_inputs': True, (TaskRunner pid=2049544) 'return_raw_chat': True, (TaskRunner pid=2049544) 'return_raw_input_ids': False, (TaskRunner pid=2049544) 'reward_fn_key': 'data_source', (TaskRunner pid=2049544) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2049544) 'seed': None, (TaskRunner pid=2049544) 'shuffle': True, (TaskRunner pid=2049544) 'tokenizer': None, (TaskRunner pid=2049544) 'tool_config_path': None, (TaskRunner pid=2049544) 'train_batch_size': 32, (TaskRunner pid=2049544) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2049544) 'train_max_samples': 3200, (TaskRunner pid=2049544) 'truncation': 'error', (TaskRunner pid=2049544) 'trust_remote_code': True, (TaskRunner pid=2049544) 'use_shm': False, (TaskRunner pid=2049544) 'val_batch_size': None, (TaskRunner pid=2049544) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2049544) 'val_max_samples': -1, (TaskRunner pid=2049544) 'validation_shuffle': False, (TaskRunner pid=2049544) 'video_key': 'videos'}, (TaskRunner pid=2049544) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2049544) 'cuda-memory-usage': 'true', (TaskRunner pid=2049544) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2049544) 'capture-range-end': None, (TaskRunner pid=2049544) 'cuda-graph-trace': 'graph', (TaskRunner pid=2049544) 'cuda-memory-usage': 'true', (TaskRunner pid=2049544) 'kill': 'none', (TaskRunner pid=2049544) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2049544) 'torch_memory': {'context': 'all', (TaskRunner pid=2049544) 'kw_args': {}, (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'stacks': 'all', (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2049544) 'profile_continuous_steps': False, (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'steps': None, (TaskRunner pid=2049544) 'tool': None}, (TaskRunner pid=2049544) 'max_model_len': 10240, (TaskRunner pid=2049544) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_grpo_Qwen3_4B', (TaskRunner pid=2049544) 'include_dashboard': False, (TaskRunner pid=2049544) 'num_cpus': None, (TaskRunner pid=2049544) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2049544) 'USER': 'root'}}}, (TaskRunner pid=2049544) 'timeline_json_file': None}, (TaskRunner pid=2049544) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2049544) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2049544) 'name': 'custom_reward_manager', (TaskRunner pid=2049544) 'path': None}, (TaskRunner pid=2049544) 'name': 'naive', (TaskRunner pid=2049544) 'source': 'register'}, (TaskRunner pid=2049544) 'reward_model': {'enable': False, (TaskRunner pid=2049544) 'enable_resource_pool': False, (TaskRunner pid=2049544) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2049544) 'launch_reward_fn_async': False, (TaskRunner pid=2049544) 'max_length': None, (TaskRunner pid=2049544) 'micro_batch_size': None, (TaskRunner pid=2049544) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2049544) 'model': {'external_lib': None, (TaskRunner pid=2049544) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2049544) 'forward_prefetch': False, (TaskRunner pid=2049544) 'fsdp_size': -1, (TaskRunner pid=2049544) 'param_offload': False, (TaskRunner pid=2049544) 'reshard_after_forward': True, (TaskRunner pid=2049544) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2049544) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2049544) 'override_config': {}, (TaskRunner pid=2049544) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2049544) 'trust_remote_code': False, (TaskRunner pid=2049544) 'use_fused_kernels': False, (TaskRunner pid=2049544) 'use_remove_padding': False, (TaskRunner pid=2049544) 'use_shm': False}, (TaskRunner pid=2049544) 'n_gpus_per_node': 8, (TaskRunner pid=2049544) 'nnodes': 0, (TaskRunner pid=2049544) 'num_workers': 1, (TaskRunner pid=2049544) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2049544) 'all_ranks': False, (TaskRunner pid=2049544) 'enable': False, (TaskRunner pid=2049544) 'ranks': [], (TaskRunner pid=2049544) 'save_path': 'outputs/profile', (TaskRunner pid=2049544) 'tool': None, (TaskRunner pid=2049544) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2049544) 'analysis': True, (TaskRunner pid=2049544) 'contents': [], (TaskRunner pid=2049544) 'discrete': False, (TaskRunner pid=2049544) 'level': 'level0'}, (TaskRunner pid=2049544) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2049544) 'discrete': False}, (TaskRunner pid=2049544) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2049544) 'step_end': None, (TaskRunner pid=2049544) 'step_start': 0}, (TaskRunner pid=2049544) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2049544) 'stack_depth': 32, (TaskRunner pid=2049544) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2049544) 'reward_loop_class_name': None, (TaskRunner pid=2049544) 'reward_loop_module_path': None, (TaskRunner pid=2049544) 'reward_loop_source': 'register', (TaskRunner pid=2049544) 'reward_manager': 'naive', (TaskRunner pid=2049544) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2049544) 'cudagraph_capture_sizes': None, (TaskRunner pid=2049544) 'data_parallel_size': 1, (TaskRunner pid=2049544) 'disable_log_stats': True, (TaskRunner pid=2049544) 'dtype': 'bfloat16', (TaskRunner pid=2049544) 'enable_chunked_prefill': True, (TaskRunner pid=2049544) 'enable_prefix_caching': True, (TaskRunner pid=2049544) 'enforce_eager': True, (TaskRunner pid=2049544) 'engine_kwargs': {}, (TaskRunner pid=2049544) 'expert_parallel_size': 1, (TaskRunner pid=2049544) 'free_cache_engine': True, (TaskRunner pid=2049544) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2049544) 'limit_images': None, (TaskRunner pid=2049544) 'load_format': 'auto', (TaskRunner pid=2049544) 'max_model_len': None, (TaskRunner pid=2049544) 'max_num_batched_tokens': 8192, (TaskRunner pid=2049544) 'max_num_seqs': 1024, (TaskRunner pid=2049544) 'name': '???', (TaskRunner pid=2049544) 'prompt_length': 2048, (TaskRunner pid=2049544) 'response_length': 2048, (TaskRunner pid=2049544) 'skip_tokenizer_init': False, (TaskRunner pid=2049544) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2049544) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2049544) 'memory_limit_mb': 1024, (TaskRunner pid=2049544) 'url': None}, (TaskRunner pid=2049544) 'strategy': 'fsdp', (TaskRunner pid=2049544) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2049544) 'use_dynamic_bsz': False, (TaskRunner pid=2049544) 'use_reward_loop': False}, (TaskRunner pid=2049544) 'trainer': {'balance_batch': True, (TaskRunner pid=2049544) 'critic_warmup': 0, (TaskRunner pid=2049544) 'default_hdfs_dir': None, (TaskRunner pid=2049544) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'del_local_ckpt_after_load': False, (TaskRunner pid=2049544) 'device': 'cuda', (TaskRunner pid=2049544) 'esi_redundant_time': 0, (TaskRunner pid=2049544) 'experiment_name': 'qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2049544) 'group_name': 'QWEN3-GRPO-generalization', (TaskRunner pid=2049544) 'log_val_generations': 0, (TaskRunner pid=2049544) 'logger': ['console', 'wandb'], (TaskRunner pid=2049544) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2049544) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2049544) 'n_gpus_per_node': 8, (TaskRunner pid=2049544) 'nnodes': 1, (TaskRunner pid=2049544) 'project_name': 'SDPO-root', (TaskRunner pid=2049544) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2049544) 'resume_from_path': None, (TaskRunner pid=2049544) 'resume_mode': 'auto', (TaskRunner pid=2049544) 'rollout_data_dir': None, (TaskRunner pid=2049544) 'save_freq': 10, (TaskRunner pid=2049544) 'test_freq': 10, (TaskRunner pid=2049544) 'total_epochs': 30, (TaskRunner pid=2049544) 'total_training_steps': 100, (TaskRunner pid=2049544) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2049544) 'val_before_train': False, (TaskRunner pid=2049544) 'val_only': False, (TaskRunner pid=2049544) 'validation_data_dir': None}, (TaskRunner pid=2049544) 'transfer_queue': {'enable': False}, (TaskRunner pid=2049544) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2049544) 'dir': '/users/root/SDPO', (TaskRunner pid=2049544) 'log_dir': '/users/root/output', (TaskRunner pid=2049544) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2049544) use_critic=need_critic(config), (TaskRunner pid=2049544) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2049544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2049544) Using dataset class: RLHFDataset (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2049544) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2049544) dataset len: 720 (TaskRunner pid=2049544) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2049544) WARNING:2026-07-02 07:32:33,214:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2049544) bind role actor_rollout method chat_completion to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method generate to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method sleep to class .WorkerDict'> (TaskRunner pid=2049544) bind role actor_rollout method wake_up to class .WorkerDict'> (TaskRunner pid=2049544) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 121.99 examples/s] (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2049544) self.use_critic = need_critic(self.config) (WorkerDict pid=2049933) [W702 07:32:42.949021168 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59177 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2049932) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2049932) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2049934) [W702 07:32:42.120202714 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:59177 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2049932) Model config after override: Qwen3Config { (WorkerDict pid=2049932) "architectures": [ (WorkerDict pid=2049932) "Qwen3ForCausalLM" (WorkerDict pid=2049932) ], (WorkerDict pid=2049932) "attention_bias": false, (WorkerDict pid=2049932) "attention_dropout": 0.0, (WorkerDict pid=2049932) "dtype": "bfloat16", (WorkerDict pid=2049932) "eos_token_id": 151645, (WorkerDict pid=2049932) "head_dim": 128, (WorkerDict pid=2049932) "hidden_act": "silu", (WorkerDict pid=2049932) "hidden_size": 2560, (WorkerDict pid=2049932) "initializer_range": 0.02, (WorkerDict pid=2049932) "intermediate_size": 9728, (WorkerDict pid=2049932) "layer_types": [ (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention", (WorkerDict pid=2049932) "full_attention" (WorkerDict pid=2049932) ], (WorkerDict pid=2049932) "max_position_embeddings": 40960, (WorkerDict pid=2049932) "max_window_layers": 36, (WorkerDict pid=2049932) "model_type": "qwen3", (WorkerDict pid=2049932) "num_attention_heads": 32, (WorkerDict pid=2049932) "num_hidden_layers": 36, (WorkerDict pid=2049932) "num_key_value_heads": 8, (WorkerDict pid=2049932) "pad_token_id": 151643, (WorkerDict pid=2049932) "rms_norm_eps": 1e-06, (WorkerDict pid=2049932) "rope_scaling": null, (WorkerDict pid=2049932) "rope_theta": 1000000, (WorkerDict pid=2049932) "sliding_window": null, (WorkerDict pid=2049932) "tie_word_embeddings": true, (WorkerDict pid=2049932) "transformers_version": "4.57.1", (WorkerDict pid=2049932) "use_cache": true, (WorkerDict pid=2049932) "use_sliding_window": false, (WorkerDict pid=2049932) "vocab_size": 151936 (WorkerDict pid=2049932) } (WorkerDict pid=2049932) (WorkerDict pid=2049932) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2049932) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2049932) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2049932) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2049932) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2049932) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2049932) Actor use_remove_padding=True (WorkerDict pid=2049932) Actor use_fused_kernels=False (WorkerDict pid=2049932) Actor use_prefix_grouper=False (WorkerDict pid=2049939) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2049939) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2049932) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2049939) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.92s/it] [repeated 12x across cluster] (WorkerDict pid=2049939) Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.97s/it] [repeated 4x across cluster] (WorkerDict pid=2049938) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.88s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.94s/it] [repeated 2x across cluster] (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2049932) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2049939) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2049939) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2049932) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2049933) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2049933) warnings.warn( (WorkerDict pid=2049939) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 7x across cluster] (WorkerDict pid=2049939) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 7x across cluster] (TaskRunner pid=2049544) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2049544) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2049932) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . [repeated 7x across cluster] (WorkerDict pid=2049932) warnings.warn( [repeated 7x across cluster] (vLLMHttpServer pid=2050805) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2050805) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2050806) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2050806) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2050805) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2050805) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2050803) ['serve', (vLLMHttpServer pid=2050803) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2050803) '--dtype', (vLLMHttpServer pid=2050803) 'bfloat16', (vLLMHttpServer pid=2050803) '--load_format', (vLLMHttpServer pid=2050803) 'dummy', (vLLMHttpServer pid=2050803) '--trust_remote_code', (vLLMHttpServer pid=2050803) '--max_model_len', (vLLMHttpServer pid=2050803) '40960', (vLLMHttpServer pid=2050803) '--max_num_seqs', (vLLMHttpServer pid=2050803) '1024', (vLLMHttpServer pid=2050803) '--enable_chunked_prefill', (vLLMHttpServer pid=2050803) '--max_num_batched_tokens', (vLLMHttpServer pid=2050803) '10240', (vLLMHttpServer pid=2050803) '--enable_prefix_caching', (vLLMHttpServer pid=2050803) '--enable_sleep_mode', (vLLMHttpServer pid=2050803) '--logprobs_mode', (vLLMHttpServer pid=2050803) 'processed_logprobs', (vLLMHttpServer pid=2050803) '--disable_custom_all_reduce', (vLLMHttpServer pid=2050803) '--gpu_memory_utilization', (vLLMHttpServer pid=2050803) '0.8', (vLLMHttpServer pid=2050803) '--disable_log_stats', (vLLMHttpServer pid=2050803) '--tensor_parallel_size', (vLLMHttpServer pid=2050803) '2', (vLLMHttpServer pid=2050803) '--seed', (vLLMHttpServer pid=2050803) '0', (vLLMHttpServer pid=2050803) '--override_generation_config', (vLLMHttpServer pid=2050803) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2050803) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2050803) '--hf_overrides', (vLLMHttpServer pid=2050803) '{}', (vLLMHttpServer pid=2050803) '--scheduling_policy', (vLLMHttpServer pid=2050803) 'fcfs'] (WorkerDict pid=2049938) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (vLLMHttpServer pid=2050805) WARNING 07-02 07:33:26 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2049936) WARNING 07-02 07:33:34 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2050803) WARNING 07-02 07:33:27 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2049934) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2049934) 2026-07-02 07:33:49,587 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2049934) 2026-07-02 07:33:49,606 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (vLLMHttpServer pid=2050803) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 2x across cluster] (vLLMHttpServer pid=2050803) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 2x across cluster] (vLLMHttpServer pid=2050803) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2050803) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2049932) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00 (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substituting the given values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Converting to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is B: 56 pC. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10 (TaskRunner pid=2049544) step:10 - global_seqlen/min:15442 - global_seqlen/max:25018 - global_seqlen/minmax_diff:9576 - global_seqlen/balanced_min:21213 - global_seqlen/balanced_max:21224 - global_seqlen/mean:21220.5 - actor/entropy:0.12735222280025482 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.40465062856674194 - training/rollout_probs_diff_mean:0.0038589579053223133 - training/rollout_probs_diff_std:0.013698565773665905 - training/rollout_actor_probs_pearson_corr:0.9970982670783997 - rollout_corr/rollout_is_mean:1.0000947713851929 - rollout_corr/rollout_is_ratio_fraction_high:3.965264113503508e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00020817638142034411 - rollout_corr/rollout_is_max:3.167215347290039 - rollout_corr/rollout_is_min:0.19278468191623688 - rollout_corr/rollout_is_std:0.03876073658466339 - rollout_corr/rollout_is_eff_sample_size:0.9985000968313013 - rollout_corr/rollout_is_seq_mean:1.000048041343689 - rollout_corr/rollout_is_seq_std:0.002764123724773526 - rollout_corr/rollout_is_seq_max:1.0100802183151245 - rollout_corr/rollout_is_seq_min:0.9903356432914734 - rollout_corr/rollout_is_seq_max_deviation:0.010080218315124512 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1353368978016078) - rollout_corr/training_log_ppl:np.float64(0.12389787597203394) - rollout_corr/kl:np.float64(0.000877203190441378) - rollout_corr/k3_kl:np.float64(0.0008888360377170557) - rollout_corr/rollout_ppl:np.float64(1.134331472683698) - rollout_corr/rollout_log_ppl:np.float64(0.12302067201380851) - rollout_corr/log_ppl_diff:np.float64(0.0008772039582254365) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020878618815913796) - rollout_corr/log_ppl_diff_max:np.float64(0.012568935751914978) - rollout_corr/log_ppl_diff_min:np.float64(-0.007140792906284332) - rollout_corr/ppl_ratio:np.float64(1.000881330575794) - rollout_corr/chi2_token:np.float64(0.0018501763697713614) - rollout_corr/chi2_seq:np.float64(0.666111497906968) - actor/pg_loss:np.float64(9.953684639185667e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005019576683480409) - actor/ppo_kl:np.float64(2.3842763525294686e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3116539753973484) - perf/mfu/actor:np.float64(0.0619170865443955) - perf/max_memory_allocated_gb:np.float64(116.3979811668396) - perf/max_memory_reserved_gb:np.float64(120.6328125) - perf/cpu_memory_used_gb:np.float64(98.92141723632812) - actor/lr:np.float64(9e-07) - val-aux/sciknoweval/reward/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.1013687608239918) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.777675) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.047588411442781434) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6271749999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07429798598036827) - val-aux/sciknoweval/score/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/score/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.1013687608239918) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.777675) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.047588411442781434) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6271749999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07429798598036827) - val-core/sciknoweval/acc/mean@16:np.float64(0.59453125) - val-aux/sciknoweval/acc/std@16:np.float64(0.1688854331093934) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6633625000000001) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13606702193420137) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5265375000000001) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14690490565059572) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.595925) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.16812839582698166) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7161875) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09695252008325414) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.46636249999999996) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1154193735626829) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6132000000000001) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13566324564246196) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.752675) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06479543702598764) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.41965) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08082900337088106) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.622625) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.1013687608239918) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.777675) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03976601626053729) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.38728750000000006) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.047588411442781434) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6271749999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07429798598036827) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.99609375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.015128841196122723) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9997875) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.0035548642615965665) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.992525) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0202257687927191) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.996175) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.014896175746267976) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.986025) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.026012467615322592) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.998875) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.008222303730994859) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9748249999999998) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.030646040504587458) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9996625) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.0039815903242650785) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9594374999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.02978871055390577) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999500000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0010787124614257437) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:10 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008162248879671097 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008162248879671097 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:394.046875 - response_length/max:1270.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:394.046875 - response_length_non_aborted/max:1270.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.09375 - prompt_length/max:382.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013142451643943787 - timing_s/agent_loop/generate_sequences/min:np.float64(0.941465875133872) - timing_s/agent_loop/generate_sequences/max:np.float64(8.937132587656379) - timing_s/agent_loop/generate_sequences/mean:np.float64(3.961100311316841) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.937132587656379) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1270 - timing_s/gen:14.806582853198051 - timing_s/reward:0.059918224811553955 - timing_s/old_log_prob:2.401917176321149 - timing_s/adv:0.15370049141347408 - timing_s/update_actor:9.923850152641535 - timing_s/step:27.43245600350201 - timing_s/testing:88.77899980731308 - timing_s/save_checkpoint:6.549710331484675 - timing_s/stop_profile:0.00011613219976425171 - timing_per_token_ms/update_actor:0.058456740844004235 - timing_per_token_ms/gen:0.14678003542168655 - timing_per_token_ms/adv:0.0009053774146077736 - perf/total_num_tokens:169764 - perf/time_per_step:27.43245600350201 - perf/throughput:773.5545077440753 (TaskRunner pid=2049544) Training Progress: 10%|█ | 10/100 [06:47<1:27:44, 58.49s/it] (TaskRunner pid=2049544) step:11 - global_seqlen/min:16821 - global_seqlen/max:31114 - global_seqlen/minmax_diff:14293 - global_seqlen/balanced_min:23187 - global_seqlen/balanced_max:29684 - global_seqlen/mean:24008.625 - actor/entropy:0.11019539088010788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5948736071586609 - training/rollout_probs_diff_mean:0.003139072796329856 - training/rollout_probs_diff_std:0.012617148458957672 - training/rollout_actor_probs_pearson_corr:0.997302770614624 - rollout_corr/rollout_is_mean:1.0000996589660645 - rollout_corr/rollout_is_ratio_fraction_high:7.389344682451338e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.852459334069863e-05 - rollout_corr/rollout_is_max:4.887231826782227 - rollout_corr/rollout_is_min:0.26104938983917236 - rollout_corr/rollout_is_std:0.03467484563589096 - rollout_corr/rollout_is_eff_sample_size:0.9987993368491189 - rollout_corr/rollout_is_seq_mean:1.0002052783966064 - rollout_corr/rollout_is_seq_std:0.0023445014376193285 - rollout_corr/rollout_is_seq_max:1.0106254816055298 - rollout_corr/rollout_is_seq_min:0.99307781457901 - rollout_corr/rollout_is_seq_max_deviation:0.010625481605529785 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1195416245609522) - rollout_corr/training_log_ppl:np.float64(0.1096741780929733) - rollout_corr/kl:np.float64(0.0007273018066071835) - rollout_corr/k3_kl:np.float64(0.0009105393462505162) - rollout_corr/rollout_ppl:np.float64(1.1187340491451323) - rollout_corr/rollout_log_ppl:np.float64(0.10894687551262905) - rollout_corr/log_ppl_diff:np.float64(0.0007273025803442579) - rollout_corr/log_ppl_abs_diff:np.float64(0.0019026359186682384) - rollout_corr/log_ppl_diff_max:np.float64(0.022861547768115997) - rollout_corr/log_ppl_diff_min:np.float64(-0.007030755281448364) - rollout_corr/ppl_ratio:np.float64(1.000731429317966) - rollout_corr/chi2_token:np.float64(0.002295879879966378) - rollout_corr/chi2_seq:np.float64(2.1700741790700704) - actor/pg_loss:np.float64(8.732720743864775e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0011307528586712579) - actor/ppo_kl:np.float64(0.00020700877354862257) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.46738822013139725) - perf/mfu/actor:np.float64(0.06503294604815439) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.65916442871094) - actor/lr:np.float64(1e-06) - training/global_step:11 - training/epoch:0 - critic/score/mean:0.609375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.609375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01939600333571434 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01939600333571434 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:475.76953125 - response_length/max:8192.0 - response_length/min:69.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:475.76953125 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:274.5 - prompt_length/max:437.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001276358962059021 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8453756030648947) - timing_s/agent_loop/generate_sequences/max:np.float64(51.0958620775491) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.681928445308586) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(51.0958620775491) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:56.71189962141216 - timing_s/reward:0.06205694004893303 - timing_s/old_log_prob:2.802142111584544 - timing_s/adv:0.1481334324926138 - timing_s/update_actor:10.832650827243924 - timing_s/step:70.63796724379063 - timing_s/stop_profile:0.00011838413774967194 - timing_per_token_ms/update_actor:0.05639978771818421 - timing_per_token_ms/gen:0.46562640805120126 - timing_per_token_ms/adv:0.0007712511258590079 - perf/total_num_tokens:192069 - perf/time_per_step:70.63796724379063 - perf/throughput:339.8827278981539 (TaskRunner pid=2049544) Training Progress: 11%|█ | 11/100 [07:58<1:32:18, 62.23s/it] (TaskRunner pid=2049544) step:12 - global_seqlen/min:18163 - global_seqlen/max:27095 - global_seqlen/minmax_diff:8932 - global_seqlen/balanced_min:23218 - global_seqlen/balanced_max:23238 - global_seqlen/mean:23234.125 - actor/entropy:0.11550334095954895 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4828122854232788 - training/rollout_probs_diff_mean:0.0033159777522087097 - training/rollout_probs_diff_std:0.012482362799346447 - training/rollout_actor_probs_pearson_corr:0.9973734021186829 - rollout_corr/rollout_is_mean:0.9999693036079407 - rollout_corr/rollout_is_ratio_fraction_high:5.898561721551232e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010111819574376568 - rollout_corr/rollout_is_max:2.5878732204437256 - rollout_corr/rollout_is_min:0.125954732298851 - rollout_corr/rollout_is_std:0.03412209078669548 - rollout_corr/rollout_is_eff_sample_size:0.9988369179946218 - rollout_corr/rollout_is_seq_mean:1.0000054836273193 - rollout_corr/rollout_is_seq_std:0.0017772279679775238 - rollout_corr/rollout_is_seq_max:1.008151888847351 - rollout_corr/rollout_is_seq_min:0.9952659010887146 - rollout_corr/rollout_is_seq_max_deviation:0.008151888847351074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1121750511229038) - rollout_corr/training_log_ppl:np.float64(0.10409450641964213) - rollout_corr/kl:np.float64(0.0006273885898906428) - rollout_corr/k3_kl:np.float64(0.000663624501786586) - rollout_corr/rollout_ppl:np.float64(1.1114575085230172) - rollout_corr/rollout_log_ppl:np.float64(0.10346711749298265) - rollout_corr/log_ppl_diff:np.float64(0.0006273889266594779) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014779563243791927) - rollout_corr/log_ppl_diff_max:np.float64(0.006399750709533691) - rollout_corr/log_ppl_diff_min:np.float64(-0.006532169878482819) - rollout_corr/ppl_ratio:np.float64(1.000629240879789) - rollout_corr/chi2_token:np.float64(0.0014743746723979712) - rollout_corr/chi2_seq:np.float64(1.0935420689638704) - actor/pg_loss:np.float64(0.00015071697998791933) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004757110639275197) - actor/ppo_kl:np.float64(2.2654555305479107e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2736458256840706) - perf/mfu/actor:np.float64(0.0671891188671343) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.26230239868164) - actor/lr:np.float64(1e-06) - training/global_step:12 - training/epoch:0 - critic/score/mean:0.59375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0065000043250620365 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0065000043250620365 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:463.56640625 - response_length/max:1365.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:463.56640625 - response_length_non_aborted/max:1365.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.5 - prompt_length/max:364.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.028241038322449e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0592438988387585) - timing_s/agent_loop/generate_sequences/max:np.float64(10.160819062963128) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.8243368406328955) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.160819062963128) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1365 - timing_s/gen:15.931704020127654 - timing_s/reward:0.06278514675796032 - timing_s/old_log_prob:2.4699926115572453 - timing_s/adv:0.16309965774416924 - timing_s/update_actor:10.090087844058871 - timing_s/step:28.80461708456278 - timing_s/stop_profile:0.0001237243413925171 - timing_per_token_ms/update_actor:0.054284849569646326 - timing_per_token_ms/gen:0.13424876779155878 - timing_per_token_ms/adv:0.0008774790192452332 - perf/total_num_tokens:185873 - perf/time_per_step:28.80461708456278 - perf/throughput:806.6111391722626 (TaskRunner pid=2049544) Training Progress: 12%|█▏ | 12/100 [08:26<1:16:22, 52.08s/it] (TaskRunner pid=2049544) step:13 - global_seqlen/min:20734 - global_seqlen/max:28493 - global_seqlen/minmax_diff:7759 - global_seqlen/balanced_min:23965 - global_seqlen/balanced_max:23977 - global_seqlen/mean:23970.125 - actor/entropy:0.1322651505470276 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5841533541679382 - training/rollout_probs_diff_mean:0.003578777192160487 - training/rollout_probs_diff_std:0.01281787734478712 - training/rollout_actor_probs_pearson_corr:0.997634768486023 - rollout_corr/rollout_is_mean:0.9998666048049927 - rollout_corr/rollout_is_ratio_fraction_high:8.307234566018451e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00010799404844874516 - rollout_corr/rollout_is_max:2.1130197048187256 - rollout_corr/rollout_is_min:0.289154976606369 - rollout_corr/rollout_is_std:0.03564285859465599 - rollout_corr/rollout_is_eff_sample_size:0.9987307228760026 - rollout_corr/rollout_is_seq_mean:0.9998607635498047 - rollout_corr/rollout_is_seq_std:0.0017514645587652922 - rollout_corr/rollout_is_seq_max:1.0048136711120605 - rollout_corr/rollout_is_seq_min:0.9948503971099854 - rollout_corr/rollout_is_seq_max_deviation:0.0051496028900146484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1441420195624232) - rollout_corr/training_log_ppl:np.float64(0.12973640582640655) - rollout_corr/kl:np.float64(0.0009200233825930049) - rollout_corr/k3_kl:np.float64(0.000771395374357553) - rollout_corr/rollout_ppl:np.float64(1.1430599559098482) - rollout_corr/rollout_log_ppl:np.float64(0.12881638239923632) - rollout_corr/log_ppl_diff:np.float64(0.0009200234271702357) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016153893011505716) - rollout_corr/log_ppl_diff_max:np.float64(0.010122627019882202) - rollout_corr/log_ppl_diff_min:np.float64(-0.007734514772891998) - rollout_corr/ppl_ratio:np.float64(1.0009225921239704) - rollout_corr/chi2_token:np.float64(0.0012212155852466822) - rollout_corr/chi2_seq:np.float64(0.878853014903143) - actor/pg_loss:np.float64(-4.01295255869627e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007891653215210681) - actor/ppo_kl:np.float64(0.00011588802720652325) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.40201497077941895) - perf/mfu/actor:np.float64(0.0691041251464859) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.31333541870117) - actor/lr:np.float64(1e-06) - training/global_step:13 - training/epoch:0 - critic/score/mean:0.515625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.515625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005807795561850071 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005807795561850071 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:470.22265625 - response_length/max:1333.0 - response_length/min:126.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:470.22265625 - response_length_non_aborted/max:1333.0 - response_length_non_aborted/min:126.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.84375 - prompt_length/max:363.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014058127999305725 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4221638720482588) - timing_s/agent_loop/generate_sequences/max:np.float64(9.880332147702575) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.85110625057132) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.880332147702575) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:1333 - timing_s/gen:15.408700393512845 - timing_s/reward:0.062333619222044945 - timing_s/old_log_prob:2.5665055103600025 - timing_s/adv:0.1314475554972887 - timing_s/update_actor:10.066378388553858 - timing_s/step:28.321861196309328 - timing_s/stop_profile:0.00011406466364860535 - timing_per_token_ms/update_actor:0.05249439869709616 - timing_per_token_ms/gen:0.12800369168124182 - timing_per_token_ms/adv:0.0006854759596439772 - perf/total_num_tokens:191761 - perf/time_per_step:28.321861196309328 - perf/throughput:846.3470968187496 (TaskRunner pid=2049544) Training Progress: 13%|█▎ | 13/100 [08:55<1:05:06, 44.90s/it] (TaskRunner pid=2049544) step:14 - global_seqlen/min:17304 - global_seqlen/max:31074 - global_seqlen/minmax_diff:13770 - global_seqlen/balanced_min:23194 - global_seqlen/balanced_max:23602 - global_seqlen/mean:23258.875 - actor/entropy:0.11846989393234253 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.45436424016952515 - training/rollout_probs_diff_mean:0.003607378341257572 - training/rollout_probs_diff_std:0.013258584775030613 - training/rollout_actor_probs_pearson_corr:0.9971495866775513 - rollout_corr/rollout_is_mean:0.9998986721038818 - rollout_corr/rollout_is_ratio_fraction_high:5.1646224164869636e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014633095997851342 - rollout_corr/rollout_is_max:2.4299769401550293 - rollout_corr/rollout_is_min:0.09816896170377731 - rollout_corr/rollout_is_std:0.037202417850494385 - rollout_corr/rollout_is_eff_sample_size:0.9986175363719597 - rollout_corr/rollout_is_seq_mean:0.9998643398284912 - rollout_corr/rollout_is_seq_std:0.002263482892885804 - rollout_corr/rollout_is_seq_max:1.00752854347229 - rollout_corr/rollout_is_seq_min:0.9923214316368103 - rollout_corr/rollout_is_seq_max_deviation:0.007678568363189697 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.13216158375144) - rollout_corr/training_log_ppl:np.float64(0.12105548306863056) - rollout_corr/kl:np.float64(0.0011141863628356674) - rollout_corr/k3_kl:np.float64(0.0007862591612095571) - rollout_corr/rollout_ppl:np.float64(1.1308507332578301) - rollout_corr/rollout_log_ppl:np.float64(0.1199412965834199) - rollout_corr/log_ppl_diff:np.float64(0.0011141864852106664) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018289975014340598) - rollout_corr/log_ppl_diff_max:np.float64(0.01189817488193512) - rollout_corr/log_ppl_diff_min:np.float64(-0.006560362875461578) - rollout_corr/ppl_ratio:np.float64(1.0011174657847732) - rollout_corr/chi2_token:np.float64(0.0008673113770782948) - rollout_corr/chi2_seq:np.float64(0.27274479530751705) - actor/pg_loss:np.float64(8.668552618473768e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0010782241606648313) - actor/ppo_kl:np.float64(0.00016768844109904535) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.42187249660491943) - perf/mfu/actor:np.float64(0.06630021670401551) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.41584777832031) - actor/lr:np.float64(1e-06) - training/global_step:14 - training/epoch:0 - critic/score/mean:0.640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009940821677446365 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009940821677446365 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:453.80859375 - response_length/max:1963.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:453.80859375 - response_length_non_aborted/max:1963.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.03125 - prompt_length/max:375.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013650394976139069 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0465375203639269) - timing_s/agent_loop/generate_sequences/max:np.float64(12.73308464512229) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.650266676486353) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.73308464512229) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:301 - timing_s/agent_loop/slowest/response_length:1963 - timing_s/gen:18.954351030290127 - timing_s/reward:0.06252953223884106 - timing_s/old_log_prob:2.473697155714035 - timing_s/adv:0.14756855927407742 - timing_s/update_actor:9.925982998684049 - timing_s/step:31.649879788979888 - timing_s/stop_profile:0.00011733733117580414 - timing_per_token_ms/update_actor:0.05334513706426068 - timing_per_token_ms/gen:0.16315344119036046 - timing_per_token_ms/adv:0.0007930766173884023 - perf/total_num_tokens:186071 - perf/time_per_step:31.649879788979888 - perf/throughput:734.8803583165097 (TaskRunner pid=2049544) Training Progress: 14%|█▍ | 14/100 [09:26<58:38, 40.91s/it] (TaskRunner pid=2049544) step:15 - global_seqlen/min:15342 - global_seqlen/max:29838 - global_seqlen/minmax_diff:14496 - global_seqlen/balanced_min:23634 - global_seqlen/balanced_max:23672 - global_seqlen/mean:23657.75 - actor/entropy:0.1110260859131813 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9913476705551147 - training/rollout_probs_diff_mean:0.0030704238452017307 - training/rollout_probs_diff_std:0.01288246363401413 - training/rollout_actor_probs_pearson_corr:0.9973083138465881 - rollout_corr/rollout_is_mean:0.9999945163726807 - rollout_corr/rollout_is_ratio_fraction_high:3.303655466879718e-05 - rollout_corr/rollout_is_ratio_fraction_low:9.085052442969754e-05 - rollout_corr/rollout_is_max:3.397390604019165 - rollout_corr/rollout_is_min:0.008578135631978512 - rollout_corr/rollout_is_std:0.03370378911495209 - rollout_corr/rollout_is_eff_sample_size:0.9988653435875123 - rollout_corr/rollout_is_seq_mean:0.9999395608901978 - rollout_corr/rollout_is_seq_std:0.0021285165566951036 - rollout_corr/rollout_is_seq_max:1.016195297241211 - rollout_corr/rollout_is_seq_min:0.9913737177848816 - rollout_corr/rollout_is_seq_max_deviation:0.016195297241210938 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.116935707628727) - rollout_corr/training_log_ppl:np.float64(0.10718949149304535) - rollout_corr/kl:np.float64(0.0008361445555777891) - rollout_corr/k3_kl:np.float64(0.0006998892279455049) - rollout_corr/rollout_ppl:np.float64(1.1159698073752224) - rollout_corr/rollout_log_ppl:np.float64(0.10635334601101931) - rollout_corr/log_ppl_diff:np.float64(0.0008361454820260406) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016929644480114803) - rollout_corr/log_ppl_diff_max:np.float64(0.01026906818151474) - rollout_corr/log_ppl_diff_min:np.float64(-0.012994319200515747) - rollout_corr/ppl_ratio:np.float64(1.000839062500745) - rollout_corr/chi2_token:np.float64(0.0011396266054362059) - rollout_corr/chi2_seq:np.float64(1.5898604474496096) - actor/pg_loss:np.float64(-2.7647125534713268e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006954610362299718) - actor/ppo_kl:np.float64(0.00011087026399003719) - actor/pg_clipfrac_lower:np.float64(1.0444518920849077e-05) - actor/grad_norm:np.float64(0.29682732000947) - perf/mfu/actor:np.float64(0.06723820699139785) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(105.3052020072937) - actor/lr:np.float64(1e-06) - training/global_step:15 - training/epoch:0 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0049513536505401134 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0049513536505401134 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:472.9609375 - response_length/max:1790.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:472.9609375 - response_length_non_aborted/max:1790.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:460.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010203756392002106 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7883450742810965) - timing_s/agent_loop/generate_sequences/max:np.float64(12.01114228926599) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.683357301946671) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.01114228926599) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:1790 - timing_s/gen:17.793763728812337 - timing_s/reward:0.06301617436110973 - timing_s/old_log_prob:2.4648690931499004 - timing_s/adv:0.15438295900821686 - timing_s/update_actor:10.23603860847652 - timing_s/step:30.79816811159253 - timing_s/stop_profile:0.00011844001710414886 - timing_per_token_ms/update_actor:0.05408396090327969 - timing_per_token_ms/gen:0.14696116328988204 - timing_per_token_ms/adv:0.0008157102799728253 - perf/total_num_tokens:189262 - perf/time_per_step:30.79816811159253 - perf/throughput:768.1544536765856 (TaskRunner pid=2049544) Training Progress: 15%|█▌ | 15/100 [09:57<53:39, 37.88s/it] (TaskRunner pid=2049544) step:16 - global_seqlen/min:18400 - global_seqlen/max:27667 - global_seqlen/minmax_diff:9267 - global_seqlen/balanced_min:22148 - global_seqlen/balanced_max:22154 - global_seqlen/mean:22151.125 - actor/entropy:0.12429671734571457 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.739967942237854 - training/rollout_probs_diff_mean:0.0033937261905521154 - training/rollout_probs_diff_std:0.012556870467960835 - training/rollout_actor_probs_pearson_corr:0.9975789189338684 - rollout_corr/rollout_is_mean:1.000154972076416 - rollout_corr/rollout_is_ratio_fraction_high:2.7087302441941574e-05 - rollout_corr/rollout_is_ratio_fraction_low:8.126190368784592e-05 - rollout_corr/rollout_is_max:4.322726726531982 - rollout_corr/rollout_is_min:0.12595464289188385 - rollout_corr/rollout_is_std:0.034952208399772644 - rollout_corr/rollout_is_eff_sample_size:0.9987801905807853 - rollout_corr/rollout_is_seq_mean:1.0001484155654907 - rollout_corr/rollout_is_seq_std:0.0019322986481711268 - rollout_corr/rollout_is_seq_max:1.007652997970581 - rollout_corr/rollout_is_seq_min:0.992692768573761 - rollout_corr/rollout_is_seq_max_deviation:0.007652997970581055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.128569976426661) - rollout_corr/training_log_ppl:np.float64(0.11689590534660965) - rollout_corr/kl:np.float64(0.0006918034323035727) - rollout_corr/k3_kl:np.float64(0.0007205118381676812) - rollout_corr/rollout_ppl:np.float64(1.1277636121958494) - rollout_corr/rollout_log_ppl:np.float64(0.11620410031173378) - rollout_corr/log_ppl_diff:np.float64(0.0006918050348758698) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016668791504343972) - rollout_corr/log_ppl_diff_max:np.float64(0.011440090835094452) - rollout_corr/log_ppl_diff_min:np.float64(-0.005701005458831787) - rollout_corr/ppl_ratio:np.float64(1.000694399466738) - rollout_corr/chi2_token:np.float64(0.0014990693889558315) - rollout_corr/chi2_seq:np.float64(0.6086118693929166) - actor/pg_loss:np.float64(3.552588168531656e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0008225217134167906) - actor/ppo_kl:np.float64(0.0001884127358140475) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3684859052300453) - perf/mfu/actor:np.float64(0.05525549139638902) - perf/max_memory_allocated_gb:np.float64(123.30060195922852) - perf/max_memory_reserved_gb:np.float64(129.0703125) - perf/cpu_memory_used_gb:np.float64(118.55257987976074) - actor/lr:np.float64(1e-06) - training/global_step:16 - training/epoch:0 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014105712994933128 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014105712994933128 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:432.62890625 - response_length/max:1203.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:432.62890625 - response_length_non_aborted/max:1203.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.59375 - prompt_length/max:344.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.934604167938232e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0808530896902084) - timing_s/agent_loop/generate_sequences/max:np.float64(8.854041801765561) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.365803626016714) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(8.854041801765561) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:334 - timing_s/agent_loop/slowest/response_length:1203 - timing_s/gen:14.700905803591013 - timing_s/reward:0.06069855950772762 - timing_s/old_log_prob:2.4508451614528894 - timing_s/adv:0.14457714557647705 - timing_s/update_actor:11.594925744459033 - timing_s/step:29.03880488872528 - timing_s/stop_profile:0.00011275149881839752 - timing_per_token_ms/update_actor:0.0654307949622143 - timing_per_token_ms/gen:0.13273596023214732 - timing_per_token_ms/adv:0.0008158566753182798 - perf/total_num_tokens:177209 - perf/time_per_step:29.03880488872528 - perf/throughput:762.8111792093924 (TaskRunner pid=2049544) Training Progress: 16%|█▌ | 16/100 [10:26<49:20, 35.25s/it] (TaskRunner pid=2049544) step:17 - global_seqlen/min:17199 - global_seqlen/max:28844 - global_seqlen/minmax_diff:11645 - global_seqlen/balanced_min:22404 - global_seqlen/balanced_max:28744 - global_seqlen/mean:23204.125 - actor/entropy:0.12878382205963135 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7316876649856567 - training/rollout_probs_diff_mean:0.003660086076706648 - training/rollout_probs_diff_std:0.013964627869427204 - training/rollout_actor_probs_pearson_corr:0.9971086978912354 - rollout_corr/rollout_is_mean:0.9999353289604187 - rollout_corr/rollout_is_ratio_fraction_high:5.108600453240797e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.000221372683881782 - rollout_corr/rollout_is_max:3.039975881576538 - rollout_corr/rollout_is_min:0.002080337144434452 - rollout_corr/rollout_is_std:0.037413306534290314 - rollout_corr/rollout_is_eff_sample_size:0.9986019633462979 - rollout_corr/rollout_is_seq_mean:0.9999227523803711 - rollout_corr/rollout_is_seq_std:0.0023962773848325014 - rollout_corr/rollout_is_seq_max:1.0085341930389404 - rollout_corr/rollout_is_seq_min:0.9915060997009277 - rollout_corr/rollout_is_seq_max_deviation:0.00853419303894043 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1416860399767756) - rollout_corr/training_log_ppl:np.float64(0.1297323260405392) - rollout_corr/kl:np.float64(0.0012029753426432421) - rollout_corr/k3_kl:np.float64(0.0010000968625991646) - rollout_corr/rollout_ppl:np.float64(1.1402671793475747) - rollout_corr/rollout_log_ppl:np.float64(0.12852934912734781) - rollout_corr/log_ppl_diff:np.float64(0.0012029769131913781) - rollout_corr/log_ppl_abs_diff:np.float64(0.0022540156060131267) - rollout_corr/log_ppl_diff_max:np.float64(0.022689037024974823) - rollout_corr/log_ppl_diff_min:np.float64(-0.0066199153661727905) - rollout_corr/ppl_ratio:np.float64(1.0012089894153178) - rollout_corr/chi2_token:np.float64(0.0014632882084697485) - rollout_corr/chi2_seq:np.float64(0.5507866519037634) - actor/pg_loss:np.float64(0.0002590372459962964) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0013125582909196964) - actor/ppo_kl:np.float64(0.00025115833264877097) - actor/pg_clipfrac_lower:np.float64(7.70463520893827e-06) - actor/grad_norm:np.float64(0.3952488452196121) - perf/mfu/actor:np.float64(0.06094650606696392) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(110.15214920043945) - actor/lr:np.float64(1e-06) - training/global_step:17 - training/epoch:0 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022923780605196953 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022923780605196953 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:458.78515625 - response_length/max:8192.0 - response_length/min:68.0 - response_length/clip_ratio:0.00390625 - response_length_non_aborted/mean:458.78515625 - response_length_non_aborted/max:8192.0 - response_length_non_aborted/min:68.0 - response_length_non_aborted/clip_ratio:0.00390625 - response/aborted_ratio:0.0 - prompt_length/mean:266.34375 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017055682837963104 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8617520779371262) - timing_s/agent_loop/generate_sequences/max:np.float64(50.32327074185014) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.232911160150252) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(50.32327074185014) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:236 - timing_s/agent_loop/slowest/response_length:8192 - timing_s/gen:57.11839736439288 - timing_s/reward:0.06247155740857124 - timing_s/old_log_prob:2.831888886168599 - timing_s/adv:0.1662355735898018 - timing_s/update_actor:10.9862146563828 - timing_s/step:71.25081997551024 - timing_s/stop_profile:0.00011354684829711914 - timing_per_token_ms/update_actor:0.05918244415800423 - timing_per_token_ms/gen:0.4863251059131442 - timing_per_token_ms/adv:0.0008955065833650363 - perf/total_num_tokens:185633 - perf/time_per_step:71.25081997551024 - perf/throughput:325.6681818956685 (TaskRunner pid=2049544) Training Progress: 17%|█▋ | 17/100 [11:38<1:03:45, 46.09s/it] (TaskRunner pid=2049544) step:18 - global_seqlen/min:17313 - global_seqlen/max:30331 - global_seqlen/minmax_diff:13018 - global_seqlen/balanced_min:25566 - global_seqlen/balanced_max:25574 - global_seqlen/mean:25570.75 - actor/entropy:0.1224943995475769 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5235333442687988 - training/rollout_probs_diff_mean:0.0032062120735645294 - training/rollout_probs_diff_std:0.012459194287657738 - training/rollout_actor_probs_pearson_corr:0.9975793361663818 - rollout_corr/rollout_is_mean:0.9999059438705444 - rollout_corr/rollout_is_ratio_fraction_high:7.298523996723816e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00012407491158228368 - rollout_corr/rollout_is_max:2.7950527667999268 - rollout_corr/rollout_is_min:0.06372947990894318 - rollout_corr/rollout_is_std:0.03364891931414604 - rollout_corr/rollout_is_eff_sample_size:0.9988689117678543 - rollout_corr/rollout_is_seq_mean:0.9998814463615417 - rollout_corr/rollout_is_seq_std:0.0018376336665824056 - rollout_corr/rollout_is_seq_max:1.007022738456726 - rollout_corr/rollout_is_seq_min:0.9937419891357422 - rollout_corr/rollout_is_seq_max_deviation:0.007022738456726074 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1279504764825106) - rollout_corr/training_log_ppl:np.float64(0.11757075608329615) - rollout_corr/kl:np.float64(0.0008173328948331005) - rollout_corr/k3_kl:np.float64(0.0006817538003787149) - rollout_corr/rollout_ppl:np.float64(1.1270086439326406) - rollout_corr/rollout_log_ppl:np.float64(0.1167534222076938) - rollout_corr/log_ppl_diff:np.float64(0.000817333875602344) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016419640196545515) - rollout_corr/log_ppl_diff_max:np.float64(0.008472427725791931) - rollout_corr/log_ppl_diff_min:np.float64(-0.006251677870750427) - rollout_corr/ppl_ratio:np.float64(1.0008197950664908) - rollout_corr/chi2_token:np.float64(0.0010528604034334421) - rollout_corr/chi2_seq:np.float64(0.8530804882757366) - actor/pg_loss:np.float64(0.0001363652991130948) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0009771198135695158) - actor/ppo_kl:np.float64(8.36708865672442e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4684727042913437) - perf/mfu/actor:np.float64(0.0710728954199909) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.35284042358398) - actor/lr:np.float64(1e-06) - training/global_step:18 - training/epoch:0 - critic/score/mean:0.40234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.40234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004924679175019264 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004924679175019264 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:535.2109375 - response_length/max:1581.0 - response_length/min:115.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:535.2109375 - response_length_non_aborted/max:1581.0 - response_length_non_aborted/min:115.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.875 - prompt_length/max:349.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012665800750255585 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8409140948206186) - timing_s/agent_loop/generate_sequences/max:np.float64(11.426093036308885) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.7903019126315485) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.426093036308885) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:1581 - timing_s/gen:17.257411571219563 - timing_s/reward:0.06468412093818188 - timing_s/old_log_prob:2.512243326753378 - timing_s/adv:0.16402030177414417 - timing_s/update_actor:10.535967061296105 - timing_s/step:30.621809538453817 - timing_s/stop_profile:0.00020102225244045258 - timing_per_token_ms/update_actor:0.05150399900910271 - timing_per_token_ms/gen:0.1259536366445733 - timing_per_token_ms/adv:0.000801796494892329 - perf/total_num_tokens:204566 - perf/time_per_step:30.621809538453817 - perf/throughput:835.0502594527971 (TaskRunner pid=2049544) Training Progress: 18%|█▊ | 18/100 [12:08<56:39, 41.46s/it] (TaskRunner pid=2049544) step:19 - global_seqlen/min:17713 - global_seqlen/max:31095 - global_seqlen/minmax_diff:13382 - global_seqlen/balanced_min:23780 - global_seqlen/balanced_max:23986 - global_seqlen/mean:23818.0 - actor/entropy:0.10652168840169907 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43985795974731445 - training/rollout_probs_diff_mean:0.003031388856470585 - training/rollout_probs_diff_std:0.012130273506045341 - training/rollout_actor_probs_pearson_corr:0.9975038766860962 - rollout_corr/rollout_is_mean:1.000024437904358 - rollout_corr/rollout_is_ratio_fraction_high:5.6943903473438695e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010575295891612768 - rollout_corr/rollout_is_max:3.544985771179199 - rollout_corr/rollout_is_min:0.2985078692436218 - rollout_corr/rollout_is_std:0.03386257588863373 - rollout_corr/rollout_is_eff_sample_size:0.9988546391994405 - rollout_corr/rollout_is_seq_mean:1.0000839233398438 - rollout_corr/rollout_is_seq_std:0.002387973479926586 - rollout_corr/rollout_is_seq_max:1.0142525434494019 - rollout_corr/rollout_is_seq_min:0.9915294051170349 - rollout_corr/rollout_is_seq_max_deviation:0.014252543449401855 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1093668043613434) - rollout_corr/training_log_ppl:np.float64(0.10132705738578807) - rollout_corr/kl:np.float64(0.0006060766052620181) - rollout_corr/k3_kl:np.float64(0.0007317086579945453) - rollout_corr/rollout_ppl:np.float64(1.1086922818794847) - rollout_corr/rollout_log_ppl:np.float64(0.10072098037198884) - rollout_corr/log_ppl_diff:np.float64(0.0006060770137992222) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016289143422909547) - rollout_corr/log_ppl_diff_max:np.float64(0.014358222484588623) - rollout_corr/log_ppl_diff_min:np.float64(-0.009524352848529816) - rollout_corr/ppl_ratio:np.float64(1.0006090987008065) - rollout_corr/chi2_token:np.float64(0.001838616793975234) - rollout_corr/chi2_seq:np.float64(0.987432076362893) - actor/pg_loss:np.float64(0.0002443152479827404) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007472785102891066) - actor/ppo_kl:np.float64(-8.193314219084868e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4798855558037758) - perf/mfu/actor:np.float64(0.0686439488762442) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.47739028930664) - actor/lr:np.float64(1e-06) - training/global_step:19 - training/epoch:0 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0009253384196199477 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0009253384196199477 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:480.1875 - response_length/max:1871.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:480.1875 - response_length_non_aborted/max:1871.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.125 - prompt_length/max:339.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001460295170545578 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0999032072722912) - timing_s/agent_loop/generate_sequences/max:np.float64(12.776574049144983) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.886515708894876) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.776574049144983) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:1871 - timing_s/gen:18.46205241046846 - timing_s/reward:0.06260044127702713 - timing_s/old_log_prob:2.540384005755186 - timing_s/adv:0.14328701607882977 - timing_s/update_actor:10.04507009498775 - timing_s/step:31.340227657929063 - timing_s/stop_profile:0.00011619366705417633 - timing_per_token_ms/update_actor:0.05271785044392765 - timing_per_token_ms/gen:0.15018590077499397 - timing_per_token_ms/adv:0.0007519891262848989 - perf/total_num_tokens:190544 - perf/time_per_step:31.340227657929063 - perf/throughput:759.9817161498524 (TaskRunner pid=2049544) Training Progress: 19%|█▉ | 19/100 [12:40<51:52, 38.43s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 20} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $. We are asked to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} = \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_10/actor (TaskRunner pid=2049544) step:20 - global_seqlen/min:17554 - global_seqlen/max:28214 - global_seqlen/minmax_diff:10660 - global_seqlen/balanced_min:22414 - global_seqlen/balanced_max:22431 - global_seqlen/mean:22418.0 - actor/entropy:0.11887378990650177 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7652756571769714 - training/rollout_probs_diff_mean:0.003253082511946559 - training/rollout_probs_diff_std:0.01256453339010477 - training/rollout_actor_probs_pearson_corr:0.9974572062492371 - rollout_corr/rollout_is_mean:0.9998491406440735 - rollout_corr/rollout_is_ratio_fraction_high:2.658820267242845e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00014180374273564667 - rollout_corr/rollout_is_max:2.1670210361480713 - rollout_corr/rollout_is_min:0.20142976939678192 - rollout_corr/rollout_is_std:0.03435710072517395 - rollout_corr/rollout_is_eff_sample_size:0.9988207434840839 - rollout_corr/rollout_is_seq_mean:0.9999091625213623 - rollout_corr/rollout_is_seq_std:0.001986477058380842 - rollout_corr/rollout_is_seq_max:1.0120420455932617 - rollout_corr/rollout_is_seq_min:0.9940481185913086 - rollout_corr/rollout_is_seq_max_deviation:0.012042045593261719 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1164336544461548) - rollout_corr/training_log_ppl:np.float64(0.10743120299957809) - rollout_corr/kl:np.float64(0.0007887022689292422) - rollout_corr/k3_kl:np.float64(0.0007427555091403804) - rollout_corr/rollout_ppl:np.float64(1.1155504160560668) - rollout_corr/rollout_log_ppl:np.float64(0.1066425001481548) - rollout_corr/log_ppl_diff:np.float64(0.0007887028514232952) - rollout_corr/log_ppl_abs_diff:np.float64(0.001653945950238267) - rollout_corr/log_ppl_diff_max:np.float64(0.012162841856479645) - rollout_corr/log_ppl_diff_min:np.float64(-0.010080903768539429) - rollout_corr/ppl_ratio:np.float64(1.0007913769222796) - rollout_corr/chi2_token:np.float64(0.0014412584714591503) - rollout_corr/chi2_seq:np.float64(0.749183313222602) - actor/pg_loss:np.float64(0.00014212285168468952) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000699050789080502) - actor/ppo_kl:np.float64(5.158489911716302e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.37614476680755615) - perf/mfu/actor:np.float64(0.06503999971026932) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.41239166259766) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09981662846229708) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8320624999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04145535225486681) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6367624999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0671080518080989) - val-aux/sciknoweval/score/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/score/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09981662846229708) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8320624999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04145535225486681) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6367624999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0671080518080989) - val-core/sciknoweval/acc/mean@16:np.float64(0.61328125) - val-aux/sciknoweval/acc/std@16:np.float64(0.18125889304909915) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6852) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15465207513909812) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5414125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14903691503963334) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6131875000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18035006026777042) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7420375) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12007545620567092) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.482225) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1146577346150142) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6277625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.14023131776921524) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.78975) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09577876488826616) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.43423750000000005) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07714076320360426) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6336375000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09981662846229708) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8320624999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.07157349089044285) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4047375) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04145535225486681) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6367624999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0671080518080989) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.990625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.030556568705962538) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9989750000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.010087512303978256) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9817375) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.040950720392224096) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.990325) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03109914326301299) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9670625000000002) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.050240658088205936) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.99635) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.01916044898238421) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9449) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05467743751202757) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.999125) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.008915794300729528) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.918325) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.047861241896262015) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.999925) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0016371701590179605) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:20 - training/epoch:0 - critic/score/mean:0.64453125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.64453125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00229323236271739 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00229323236271739 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:440.75 - response_length/max:1682.0 - response_length/min:116.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:440.75 - response_length_non_aborted/max:1682.0 - response_length_non_aborted/min:116.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.8125 - prompt_length/max:355.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.746236562728882e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.339398443698883) - timing_s/agent_loop/generate_sequences/max:np.float64(11.241939969360828) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.378861661745759) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.241939969360828) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:1682 - timing_s/gen:17.086866164579988 - timing_s/reward:0.061425795778632164 - timing_s/old_log_prob:2.430392887443304 - timing_s/adv:0.2020460832864046 - timing_s/update_actor:9.794211328029633 - timing_s/step:29.6614922080189 - timing_s/testing:97.09009108878672 - timing_s/save_checkpoint:7.344661565497518 - timing_s/stop_profile:0.00012345612049102783 - timing_per_token_ms/update_actor:0.054611313052177 - timing_per_token_ms/gen:0.15143634930321173 - timing_per_token_ms/adv:0.001126584013328601 - perf/total_num_tokens:179344 - perf/time_per_step:29.6614922080189 - perf/throughput:755.7947470336425 (TaskRunner pid=2049544) Training Progress: 20%|██ | 20/100 [14:54<1:29:33, 67.17s/it] (TaskRunner pid=2049544) step:21 - global_seqlen/min:20890 - global_seqlen/max:29372 - global_seqlen/minmax_diff:8482 - global_seqlen/balanced_min:23784 - global_seqlen/balanced_max:24194 - global_seqlen/mean:23842.25 - actor/entropy:0.10979560017585754 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7417163848876953 - training/rollout_probs_diff_mean:0.0029196166433393955 - training/rollout_probs_diff_std:0.01234018336981535 - training/rollout_actor_probs_pearson_corr:0.9974682927131653 - rollout_corr/rollout_is_mean:1.0000228881835938 - rollout_corr/rollout_is_ratio_fraction_high:1.6375992345274426e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.00010644395661074668 - rollout_corr/rollout_is_max:2.094970703125 - rollout_corr/rollout_is_min:0.07465586066246033 - rollout_corr/rollout_is_std:0.03303755819797516 - rollout_corr/rollout_is_eff_sample_size:0.9989098287246259 - rollout_corr/rollout_is_seq_mean:0.9999141693115234 - rollout_corr/rollout_is_seq_std:0.0017565247835591435 - rollout_corr/rollout_is_seq_max:1.0051144361495972 - rollout_corr/rollout_is_seq_min:0.9947226643562317 - rollout_corr/rollout_is_seq_max_deviation:0.0052773356437683105 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1051466539502144) - rollout_corr/training_log_ppl:np.float64(0.09741823082003975) - rollout_corr/kl:np.float64(0.0006470407322218819) - rollout_corr/k3_kl:np.float64(0.0006291852859376945) - rollout_corr/rollout_ppl:np.float64(1.1044287369586527) - rollout_corr/rollout_log_ppl:np.float64(0.09677118871331913) - rollout_corr/log_ppl_diff:np.float64(0.0006470421067206189) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014577525143977255) - rollout_corr/log_ppl_diff_max:np.float64(0.009566932916641235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005102425813674927) - rollout_corr/ppl_ratio:np.float64(1.0006491616368294) - rollout_corr/chi2_token:np.float64(0.0012274319306015968) - rollout_corr/chi2_seq:np.float64(1.2051882504019886) - actor/pg_loss:np.float64(0.00013444910291582346) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00045100462170921674) - actor/ppo_kl:np.float64(-5.319785113400144e-05) - actor/pg_clipfrac_lower:np.float64(2.38095403801708e-05) - actor/grad_norm:np.float64(0.2096654698252678) - perf/mfu/actor:np.float64(0.0669203788543201) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.66231536865234) - actor/lr:np.float64(1e-06) - training/global_step:21 - training/epoch:0 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004104233346879482 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004104233346879482 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:477.0703125 - response_length/max:2091.0 - response_length/min:152.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:477.0703125 - response_length_non_aborted/max:2091.0 - response_length_non_aborted/min:152.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.0 - prompt_length/max:364.0 - prompt_length/min:195.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015165284276008606 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7176804710179567) - timing_s/agent_loop/generate_sequences/max:np.float64(12.88374364003539) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.737000476088724) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.88374364003539) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:2091 - timing_s/gen:19.051039807498455 - timing_s/reward:0.05670631118118763 - timing_s/old_log_prob:2.6326032262295485 - timing_s/adv:0.14211471565067768 - timing_s/update_actor:10.165599260479212 - timing_s/step:32.148173009976745 - timing_s/stop_profile:0.000135812908411026 - timing_per_token_ms/update_actor:0.053296140572299235 - timing_per_token_ms/gen:0.1559898453082654 - timing_per_token_ms/adv:0.000745078147252659 - perf/total_num_tokens:190738 - perf/time_per_step:32.148173009976745 - perf/throughput:741.6362352100347 (TaskRunner pid=2049544) Training Progress: 21%|██ | 21/100 [15:26<1:14:37, 56.68s/it] (TaskRunner pid=2049544) step:22 - global_seqlen/min:16900 - global_seqlen/max:32996 - global_seqlen/minmax_diff:16096 - global_seqlen/balanced_min:23691 - global_seqlen/balanced_max:23913 - global_seqlen/mean:23732.375 - actor/entropy:0.12188427150249481 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5312178134918213 - training/rollout_probs_diff_mean:0.0033468599431216717 - training/rollout_probs_diff_std:0.013181475922465324 - training/rollout_actor_probs_pearson_corr:0.9972920417785645 - rollout_corr/rollout_is_mean:0.999908983707428 - rollout_corr/rollout_is_ratio_fraction_high:2.4285402105306275e-05 - rollout_corr/rollout_is_ratio_fraction_low:0.0001942832168424502 - rollout_corr/rollout_is_max:3.412278175354004 - rollout_corr/rollout_is_min:0.14518168568611145 - rollout_corr/rollout_is_std:0.03405214846134186 - rollout_corr/rollout_is_eff_sample_size:0.9988416753032981 - rollout_corr/rollout_is_seq_mean:0.999866247177124 - rollout_corr/rollout_is_seq_std:0.00229041394777596 - rollout_corr/rollout_is_seq_max:1.0077131986618042 - rollout_corr/rollout_is_seq_min:0.9842400550842285 - rollout_corr/rollout_is_seq_max_deviation:0.015759944915771484 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1243417421355844) - rollout_corr/training_log_ppl:np.float64(0.11367632360634161) - rollout_corr/kl:np.float64(0.0010524871724442164) - rollout_corr/k3_kl:np.float64(0.000926966261786788) - rollout_corr/rollout_ppl:np.float64(1.1231554276309907) - rollout_corr/rollout_log_ppl:np.float64(0.11262383543362375) - rollout_corr/log_ppl_diff:np.float64(0.001052488172717858) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018730249648797326) - rollout_corr/log_ppl_diff_max:np.float64(0.0278562530875206) - rollout_corr/log_ppl_diff_min:np.float64(-0.004908844828605652) - rollout_corr/ppl_ratio:np.float64(1.0010584215633571) - rollout_corr/chi2_token:np.float64(0.0015142448246479034) - rollout_corr/chi2_seq:np.float64(0.5919819474220276) - actor/pg_loss:np.float64(-3.6185141652822495e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007750144300189277) - actor/ppo_kl:np.float64(0.00017040614556229627) - actor/pg_clipfrac_lower:np.float64(5.271592272038106e-06) - actor/grad_norm:np.float64(0.4317408576607704) - perf/mfu/actor:np.float64(0.06862934905562151) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(105.66873407363892) - actor/lr:np.float64(1e-06) - training/global_step:22 - training/epoch:0 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0038765573408454657 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0038765573408454657 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:482.54296875 - response_length/max:1955.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:482.54296875 - response_length_non_aborted/max:1955.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.09375 - prompt_length/max:375.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013677775859832764 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3632201459258795) - timing_s/agent_loop/generate_sequences/max:np.float64(12.304583990946412) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.897737120372767) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.304583990946412) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1955 - timing_s/gen:18.77624169550836 - timing_s/reward:0.054932184517383575 - timing_s/old_log_prob:2.5277608297765255 - timing_s/adv:0.14852365665137768 - timing_s/update_actor:10.204811798408628 - timing_s/step:31.823464412242174 - timing_s/stop_profile:0.0001289471983909607 - timing_per_token_ms/update_actor:0.05374942351117739 - timing_per_token_ms/gen:0.15199619282211235 - timing_per_token_ms/adv:0.0007822839931284673 - perf/total_num_tokens:189859 - perf/time_per_step:31.823464412242174 - perf/throughput:745.7508300344066 (TaskRunner pid=2049544) Training Progress: 22%|██▏ | 22/100 [15:58<1:04:00, 49.24s/it] (TaskRunner pid=2049544) step:23 - global_seqlen/min:17606 - global_seqlen/max:33708 - global_seqlen/minmax_diff:16102 - global_seqlen/balanced_min:26856 - global_seqlen/balanced_max:26886 - global_seqlen/mean:26877.375 - actor/entropy:0.1653793752193451 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9992972612380981 - training/rollout_probs_diff_mean:0.003593530971556902 - training/rollout_probs_diff_std:0.01221021730452776 - training/rollout_actor_probs_pearson_corr:0.9982529878616333 - rollout_corr/rollout_is_mean:1.0001059770584106 - rollout_corr/rollout_is_ratio_fraction_high:2.7500669602886774e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.18765116087161e-05 - rollout_corr/rollout_is_max:3.486543655395508 - rollout_corr/rollout_is_min:0.00043065936188213527 - rollout_corr/rollout_is_std:0.034376177936792374 - rollout_corr/rollout_is_eff_sample_size:0.9988199109866871 - rollout_corr/rollout_is_seq_mean:1.0000687837600708 - rollout_corr/rollout_is_seq_std:0.0017359366174787283 - rollout_corr/rollout_is_seq_max:1.0058130025863647 - rollout_corr/rollout_is_seq_min:0.9936341047286987 - rollout_corr/rollout_is_seq_max_deviation:0.0063658952713012695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1735385786741972) - rollout_corr/training_log_ppl:np.float64(0.1555803755109082) - rollout_corr/kl:np.float64(0.0008435647158400172) - rollout_corr/k3_kl:np.float64(0.000810335308230492) - rollout_corr/rollout_ppl:np.float64(1.172548221424222) - rollout_corr/rollout_log_ppl:np.float64(0.15473681054572808) - rollout_corr/log_ppl_diff:np.float64(0.0008435649651801214) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016919265181059018) - rollout_corr/log_ppl_diff_max:np.float64(0.013718657195568085) - rollout_corr/log_ppl_diff_min:np.float64(-0.005128346383571625) - rollout_corr/ppl_ratio:np.float64(1.000846435315907) - rollout_corr/chi2_token:np.float64(0.001545611536130309) - rollout_corr/chi2_seq:np.float64(0.7387706565205008) - actor/pg_loss:np.float64(-7.304479368031025e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0014301339112989808) - actor/ppo_kl:np.float64(0.00022386275868435668) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.4195772558450699) - perf/mfu/actor:np.float64(0.07791174639008124) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.34968948364258) - actor/lr:np.float64(1e-06) - training/global_step:23 - training/epoch:1 - critic/score/mean:0.46484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.46484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.014247066341340542 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.014247066341340542 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:568.16796875 - response_length/max:1852.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:568.16796875 - response_length_non_aborted/max:1852.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.75 - prompt_length/max:375.0 - prompt_length/min:203.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003154464066028595 - timing_s/agent_loop/generate_sequences/min:np.float64(1.197395345196128) - timing_s/agent_loop/generate_sequences/max:np.float64(13.042586963623762) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.787129377022211) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.042586963623762) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:1852 - timing_s/gen:18.867352159693837 - timing_s/reward:0.06311722472310066 - timing_s/old_log_prob:2.6495919916778803 - timing_s/adv:0.15099932067096233 - timing_s/update_actor:9.910499073565006 - timing_s/step:31.743287844583392 - timing_s/stop_profile:0.00010893121361732483 - timing_per_token_ms/update_actor:0.046091271346090375 - timing_per_token_ms/gen:0.12971620793046343 - timing_per_token_ms/adv:0.0007022603615074126 - perf/total_num_tokens:215019 - perf/time_per_step:31.743287844583392 - perf/throughput:846.7104961399359 (TaskRunner pid=2049544) Training Progress: 23%|██▎ | 23/100 [16:30<56:36, 44.11s/it] (TaskRunner pid=2049544) step:24 - global_seqlen/min:18041 - global_seqlen/max:30779 - global_seqlen/minmax_diff:12738 - global_seqlen/balanced_min:24022 - global_seqlen/balanced_max:24069 - global_seqlen/mean:24056.125 - actor/entropy:0.143111914396286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8866641521453857 - training/rollout_probs_diff_mean:0.003361048409715295 - training/rollout_probs_diff_std:0.012565950863063335 - training/rollout_actor_probs_pearson_corr:0.9978638887405396 - rollout_corr/rollout_is_mean:1.0000112056732178 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.00013656703231390566 - rollout_corr/rollout_is_max:1.9401601552963257 - rollout_corr/rollout_is_min:0.01988193951547146 - rollout_corr/rollout_is_std:0.03432759642601013 - rollout_corr/rollout_is_eff_sample_size:0.9988230031268704 - rollout_corr/rollout_is_seq_mean:1.000041127204895 - rollout_corr/rollout_is_seq_std:0.0024783890694379807 - rollout_corr/rollout_is_seq_max:1.0213310718536377 - rollout_corr/rollout_is_seq_min:0.9930093884468079 - rollout_corr/rollout_is_seq_max_deviation:0.021331071853637695 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.144452238921076) - rollout_corr/training_log_ppl:np.float64(0.13059414884628495) - rollout_corr/kl:np.float64(0.00082983940849779) - rollout_corr/k3_kl:np.float64(0.0008679421140129762) - rollout_corr/rollout_ppl:np.float64(1.1434855586849153) - rollout_corr/rollout_log_ppl:np.float64(0.12976430927665206) - rollout_corr/log_ppl_diff:np.float64(0.0008298395696328953) - rollout_corr/log_ppl_abs_diff:np.float64(0.001792395327356644) - rollout_corr/log_ppl_diff_max:np.float64(0.01518431305885315) - rollout_corr/log_ppl_diff_min:np.float64(-0.016911908984184265) - rollout_corr/ppl_ratio:np.float64(1.000833987724036) - rollout_corr/chi2_token:np.float64(0.0022546271793544292) - rollout_corr/chi2_seq:np.float64(1.743649244075641) - actor/pg_loss:np.float64(0.00021742633543908596) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007519598264025262) - actor/ppo_kl:np.float64(9.309058857098762e-05) - actor/pg_clipfrac_lower:np.float64(5.556543328566477e-06) - actor/grad_norm:np.float64(0.3718913719058037) - perf/mfu/actor:np.float64(0.06935155745712235) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.33251953125) - actor/lr:np.float64(1e-06) - training/global_step:24 - training/epoch:1 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.010280685499310493 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.010280685499310493 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:486.25390625 - response_length/max:1963.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:486.25390625 - response_length_non_aborted/max:1963.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.5 - prompt_length/max:383.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013114511966705322 - timing_s/agent_loop/generate_sequences/min:np.float64(1.162827044725418) - timing_s/agent_loop/generate_sequences/max:np.float64(12.901960568502545) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.848180819972185) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(12.901960568502545) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:257 - timing_s/agent_loop/slowest/response_length:1963 - timing_s/gen:18.82960532233119 - timing_s/reward:0.0641421228647232 - timing_s/old_log_prob:2.5491561722010374 - timing_s/adv:0.1262466385960579 - timing_s/update_actor:10.136537404730916 - timing_s/step:31.79429466277361 - timing_s/stop_profile:0.00012167543172836304 - timing_per_token_ms/update_actor:0.052671291639504056 - timing_per_token_ms/gen:0.15126489442028254 - timing_per_token_ms/adv:0.0006560004915383186 - perf/total_num_tokens:192449 - perf/time_per_step:31.79429466277361 - perf/throughput:756.6176653752331 (TaskRunner pid=2049544) Training Progress: 24%|██▍ | 24/100 [17:02<51:12, 40.43s/it] (TaskRunner pid=2049544) step:25 - global_seqlen/min:17209 - global_seqlen/max:32556 - global_seqlen/minmax_diff:15347 - global_seqlen/balanced_min:25066 - global_seqlen/balanced_max:25097 - global_seqlen/mean:25084.5 - actor/entropy:0.16078905761241913 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7048856616020203 - training/rollout_probs_diff_mean:0.003612553235143423 - training/rollout_probs_diff_std:0.01245187222957611 - training/rollout_actor_probs_pearson_corr:0.9980130791664124 - rollout_corr/rollout_is_mean:1.0001463890075684 - rollout_corr/rollout_is_ratio_fraction_high:3.095304418820888e-05 - rollout_corr/rollout_is_ratio_fraction_low:7.73826104705222e-05 - rollout_corr/rollout_is_max:2.4638538360595703 - rollout_corr/rollout_is_min:0.07782533764839172 - rollout_corr/rollout_is_std:0.03481380268931389 - rollout_corr/rollout_is_eff_sample_size:0.9987899420046099 - rollout_corr/rollout_is_seq_mean:1.0001909732818604 - rollout_corr/rollout_is_seq_std:0.0020113997161388397 - rollout_corr/rollout_is_seq_max:1.0118286609649658 - rollout_corr/rollout_is_seq_min:0.9921811819076538 - rollout_corr/rollout_is_seq_max_deviation:0.01182866096496582 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1733281780034304) - rollout_corr/training_log_ppl:np.float64(0.15433276955445763) - rollout_corr/kl:np.float64(0.0005177396068951623) - rollout_corr/k3_kl:np.float64(0.0007405701547043009) - rollout_corr/rollout_ppl:np.float64(1.172693099360913) - rollout_corr/rollout_log_ppl:np.float64(0.15381502895615995) - rollout_corr/log_ppl_diff:np.float64(0.0005177405982976779) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015419888804899529) - rollout_corr/log_ppl_diff_max:np.float64(0.017087146639823914) - rollout_corr/log_ppl_diff_min:np.float64(-0.010565042495727539) - rollout_corr/ppl_ratio:np.float64(1.0005205699708313) - rollout_corr/chi2_token:np.float64(0.0020056585781276226) - rollout_corr/chi2_seq:np.float64(0.9553266488946974) - actor/pg_loss:np.float64(7.516134064644575e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005029199624004832) - actor/ppo_kl:np.float64(6.852521258160493e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3206869065761566) - perf/mfu/actor:np.float64(0.07269379031279649) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.39129257202148) - actor/lr:np.float64(1e-06) - training/global_step:25 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.005469983443617821 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.005469983443617821 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:504.796875 - response_length/max:1635.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:504.796875 - response_length_non_aborted/max:1635.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.09375 - prompt_length/max:365.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001018233597278595 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8177736550569534) - timing_s/agent_loop/generate_sequences/max:np.float64(11.67398145236075) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.060982199414866) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.67398145236075) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:335 - timing_s/agent_loop/slowest/response_length:1635 - timing_s/gen:17.524463137611747 - timing_s/reward:0.05791650153696537 - timing_s/old_log_prob:2.436915224418044 - timing_s/adv:0.13883326761424541 - timing_s/update_actor:9.980426739901304 - timing_s/step:30.245057743042707 - timing_s/stop_profile:0.00012283958494663239 - timing_per_token_ms/update_actor:0.049734032669085014 - timing_per_token_ms/gen:0.13560887065970026 - timing_per_token_ms/adv:0.0006918279595678876 - perf/total_num_tokens:200676 - perf/time_per_step:30.245057743042707 - perf/throughput:829.3751730651004 (TaskRunner pid=2049544) Training Progress: 25%|██▌ | 25/100 [17:32<46:43, 37.39s/it] (TaskRunner pid=2049544) (TaskRunner pid=2049544) step:26 - global_seqlen/min:19588 - global_seqlen/max:26583 - global_seqlen/minmax_diff:6995 - global_seqlen/balanced_min:22329 - global_seqlen/balanced_max:22375 - global_seqlen/mean:22358.5 - actor/entropy:0.14037299156188965 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3580157160758972 - training/rollout_probs_diff_mean:0.003176919650286436 - training/rollout_probs_diff_std:0.011454894207417965 - training/rollout_actor_probs_pearson_corr:0.9981624484062195 - rollout_corr/rollout_is_mean:1.0003305673599243 - rollout_corr/rollout_is_ratio_fraction_high:2.6683745090849698e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.6683745090849698e-05 - rollout_corr/rollout_is_max:3.044771909713745 - rollout_corr/rollout_is_min:0.3202765882015228 - rollout_corr/rollout_is_std:0.03229307755827904 - rollout_corr/rollout_is_eff_sample_size:0.9989589572900974 - rollout_corr/rollout_is_seq_mean:1.0003631114959717 - rollout_corr/rollout_is_seq_std:0.001997264800593257 - rollout_corr/rollout_is_seq_max:1.009385347366333 - rollout_corr/rollout_is_seq_min:0.9931842684745789 - rollout_corr/rollout_is_seq_max_deviation:0.009385347366333008 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.137321101501584) - rollout_corr/training_log_ppl:np.float64(0.12318833442259347) - rollout_corr/kl:np.float64(0.00036465673569097135) - rollout_corr/k3_kl:np.float64(0.0006004141492397252) - rollout_corr/rollout_ppl:np.float64(1.136900127865374) - rollout_corr/rollout_log_ppl:np.float64(0.12282367694933782) - rollout_corr/log_ppl_diff:np.float64(0.00036465747325564735) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014688569026475307) - rollout_corr/log_ppl_diff_max:np.float64(0.011015359312295914) - rollout_corr/log_ppl_diff_min:np.float64(-0.0049578845500946045) - rollout_corr/ppl_ratio:np.float64(1.0003668149001896) - rollout_corr/chi2_token:np.float64(0.001733903307467699) - rollout_corr/chi2_seq:np.float64(0.5194167881272733) - actor/pg_loss:np.float64(8.537911344319582e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003864719383273041) - actor/ppo_kl:np.float64(0.00016390427055512635) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1358119323849678) - perf/mfu/actor:np.float64(0.06507624815359347) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.42486190795898) - actor/lr:np.float64(1e-06) - training/global_step:26 - training/epoch:1 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01127944141626358 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01127944141626358 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:439.171875 - response_length/max:1781.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:439.171875 - response_length_non_aborted/max:1781.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.53125 - prompt_length/max:363.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013301707804203033 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8018740545958281) - timing_s/agent_loop/generate_sequences/max:np.float64(11.262868203222752) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.332270985694777) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.262868203222752) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:316 - timing_s/agent_loop/slowest/response_length:1781 - timing_s/gen:17.137579491361976 - timing_s/reward:0.06755838170647621 - timing_s/old_log_prob:2.513957444578409 - timing_s/adv:0.14904913492500782 - timing_s/update_actor:10.074241016060114 - timing_s/step:30.039264272898436 - timing_s/stop_profile:0.00011388026177883148 - timing_per_token_ms/update_actor:0.05632220976396065 - timing_per_token_ms/gen:0.15243159614475021 - timing_per_token_ms/adv:0.0008332912255127122 - perf/total_num_tokens:178868 - perf/time_per_step:30.039264272898436 - perf/throughput:744.309174714773 (TaskRunner pid=2049544) Training Progress: 26%|██▌ | 26/100 [18:02<43:24, 35.20s/it] (TaskRunner pid=2049544) step:27 - global_seqlen/min:20390 - global_seqlen/max:35494 - global_seqlen/minmax_diff:15104 - global_seqlen/balanced_min:27757 - global_seqlen/balanced_max:27828 - global_seqlen/mean:27799.5 - actor/entropy:0.13566820323467255 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4101554751396179 - training/rollout_probs_diff_mean:0.002812632592394948 - training/rollout_probs_diff_std:0.010530136525630951 - training/rollout_actor_probs_pearson_corr:0.9983336925506592 - rollout_corr/rollout_is_mean:1.0000394582748413 - rollout_corr/rollout_is_ratio_fraction_high:1.9780045477091335e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.615344005287625e-05 - rollout_corr/rollout_is_max:2.5563464164733887 - rollout_corr/rollout_is_min:0.3321341276168823 - rollout_corr/rollout_is_std:0.03021036833524704 - rollout_corr/rollout_is_eff_sample_size:0.9990882848714958 - rollout_corr/rollout_is_seq_mean:1.0000697374343872 - rollout_corr/rollout_is_seq_std:0.0016781107988208532 - rollout_corr/rollout_is_seq_max:1.0076134204864502 - rollout_corr/rollout_is_seq_min:0.9905708432197571 - rollout_corr/rollout_is_seq_max_deviation:0.00942915678024292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1346074021421373) - rollout_corr/training_log_ppl:np.float64(0.12218492220563348) - rollout_corr/kl:np.float64(0.0005914641850237601) - rollout_corr/k3_kl:np.float64(0.0005361424209464616) - rollout_corr/rollout_ppl:np.float64(1.1339335390366614) - rollout_corr/rollout_log_ppl:np.float64(0.1215934576903237) - rollout_corr/log_ppl_diff:np.float64(0.0005914645153097808) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011890121677424759) - rollout_corr/log_ppl_diff_max:np.float64(0.012660838663578033) - rollout_corr/log_ppl_diff_min:np.float64(-0.006112679839134216) - rollout_corr/ppl_ratio:np.float64(1.0005930345505476) - rollout_corr/chi2_token:np.float64(0.0009534833952784538) - rollout_corr/chi2_seq:np.float64(1.2373452726751566) - actor/pg_loss:np.float64(0.00019052473362535238) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004077799751485145) - actor/ppo_kl:np.float64(0.00015187378462222512) - actor/pg_clipfrac_lower:np.float64(2.895663556046202e-06) - actor/grad_norm:np.float64(0.24880316480994225) - perf/mfu/actor:np.float64(0.08016295921139835) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.3461446762085) - actor/lr:np.float64(1e-06) - training/global_step:27 - training/epoch:1 - critic/score/mean:0.71875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0011563085718080401 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0011563085718080401 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:592.453125 - response_length/max:2410.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:592.453125 - response_length_non_aborted/max:2410.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.28125 - prompt_length/max:375.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010118260979652405 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9887365885078907) - timing_s/agent_loop/generate_sequences/max:np.float64(15.905478810891509) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.845138130702253) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.905478810891509) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:233 - timing_s/agent_loop/slowest/response_length:2410 - timing_s/gen:21.729817861691117 - timing_s/reward:0.06810113973915577 - timing_s/old_log_prob:2.5614845752716064 - timing_s/adv:0.1540636345744133 - timing_s/update_actor:10.151232639327645 - timing_s/step:34.75130227394402 - timing_s/stop_profile:0.00013370253145694733 - timing_per_token_ms/update_actor:0.04564485260223945 - timing_per_token_ms/gen:0.1432722648264045 - timing_per_token_ms/adv:0.0006927446292847592 - perf/total_num_tokens:222396 - perf/time_per_step:34.75130227394402 - perf/throughput:799.9556327661317 (TaskRunner pid=2049544) Training Progress: 27%|██▋ | 27/100 [18:37<42:40, 35.08s/it] (TaskRunner pid=2049544) step:28 - global_seqlen/min:17957 - global_seqlen/max:34244 - global_seqlen/minmax_diff:16287 - global_seqlen/balanced_min:25513 - global_seqlen/balanced_max:25550 - global_seqlen/mean:25526.75 - actor/entropy:0.16894282400608063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47068703174591064 - training/rollout_probs_diff_mean:0.003456175560131669 - training/rollout_probs_diff_std:0.011879513040184975 - training/rollout_actor_probs_pearson_corr:0.9982811212539673 - rollout_corr/rollout_is_mean:0.9999144077301025 - rollout_corr/rollout_is_ratio_fraction_high:1.4930944416846614e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.972377766738646e-05 - rollout_corr/rollout_is_max:2.6947081089019775 - rollout_corr/rollout_is_min:0.39785826206207275 - rollout_corr/rollout_is_std:0.03413606435060501 - rollout_corr/rollout_is_eff_sample_size:0.9988359665383245 - rollout_corr/rollout_is_seq_mean:0.9999093413352966 - rollout_corr/rollout_is_seq_std:0.002044897060841322 - rollout_corr/rollout_is_seq_max:1.0098328590393066 - rollout_corr/rollout_is_seq_min:0.9918521642684937 - rollout_corr/rollout_is_seq_max_deviation:0.00983285903930664 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1787242372520268) - rollout_corr/training_log_ppl:np.float64(0.15832054920610972) - rollout_corr/kl:np.float64(0.0006220183064016283) - rollout_corr/k3_kl:np.float64(0.0006294625401039866) - rollout_corr/rollout_ppl:np.float64(1.1779772732406855) - rollout_corr/rollout_log_ppl:np.float64(0.15769852932862705) - rollout_corr/log_ppl_diff:np.float64(0.0006220198774826713) - rollout_corr/log_ppl_abs_diff:np.float64(0.001586154270626139) - rollout_corr/log_ppl_diff_max:np.float64(0.009446874260902405) - rollout_corr/log_ppl_diff_min:np.float64(-0.006108276546001434) - rollout_corr/ppl_ratio:np.float64(1.0006244094111025) - rollout_corr/chi2_token:np.float64(0.0012534137349575758) - rollout_corr/chi2_seq:np.float64(0.5095949505921453) - actor/pg_loss:np.float64(0.0001528579741716385) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005431991780824319) - actor/ppo_kl:np.float64(-0.00011134325540240742) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2734988182783127) - perf/mfu/actor:np.float64(0.07474159440863055) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.44351959228516) - actor/lr:np.float64(1e-06) - training/global_step:28 - training/epoch:1 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018798992037773132 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018798992037773132 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:523.2421875 - response_length/max:1995.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:523.2421875 - response_length_non_aborted/max:1995.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:274.46875 - prompt_length/max:363.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001253858208656311 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8268284611403942) - timing_s/agent_loop/generate_sequences/max:np.float64(13.178542386740446) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.144354823620233) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.178542386740446) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:302 - timing_s/agent_loop/slowest/response_length:1995 - timing_s/gen:19.040658405050635 - timing_s/reward:0.06457959674298763 - timing_s/old_log_prob:2.444152580574155 - timing_s/adv:0.15133203007280827 - timing_s/update_actor:10.063168801367283 - timing_s/step:31.856078814715147 - timing_s/stop_profile:0.00013601407408714294 - timing_per_token_ms/update_actor:0.04927756569758823 - timing_per_token_ms/gen:0.14214750582344632 - timing_per_token_ms/adv:0.0007410463047235168 - perf/total_num_tokens:204214 - perf/time_per_step:31.856078814715147 - perf/throughput:801.3148808574813 (TaskRunner pid=2049544) Training Progress: 28%|██▊ | 28/100 [19:09<40:57, 34.13s/it] (TaskRunner pid=2049544) step:29 - global_seqlen/min:20947 - global_seqlen/max:33732 - global_seqlen/minmax_diff:12785 - global_seqlen/balanced_min:25386 - global_seqlen/balanced_max:25685 - global_seqlen/mean:25470.375 - actor/entropy:0.13968856632709503 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5444822311401367 - training/rollout_probs_diff_mean:0.0031616955529898405 - training/rollout_probs_diff_std:0.011410399340093136 - training/rollout_actor_probs_pearson_corr:0.9981111884117126 - rollout_corr/rollout_is_mean:1.000052809715271 - rollout_corr/rollout_is_ratio_fraction_high:7.336810995184351e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.8694487961474806e-05 - rollout_corr/rollout_is_max:3.6677300930023193 - rollout_corr/rollout_is_min:0.22905343770980835 - rollout_corr/rollout_is_std:0.0324532613158226 - rollout_corr/rollout_is_eff_sample_size:0.9989480129782379 - rollout_corr/rollout_is_seq_mean:1.0001238584518433 - rollout_corr/rollout_is_seq_std:0.002332444768399 - rollout_corr/rollout_is_seq_max:1.0094679594039917 - rollout_corr/rollout_is_seq_min:0.9920924305915833 - rollout_corr/rollout_is_seq_max_deviation:0.0094679594039917 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.140123389661312) - rollout_corr/training_log_ppl:np.float64(0.12807749381863687) - rollout_corr/kl:np.float64(0.000638476259126719) - rollout_corr/k3_kl:np.float64(0.0006935356665564996) - rollout_corr/rollout_ppl:np.float64(1.1393897044472396) - rollout_corr/rollout_log_ppl:np.float64(0.1274390167054662) - rollout_corr/log_ppl_diff:np.float64(0.0006384771131706657) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017721431377140107) - rollout_corr/log_ppl_diff_max:np.float64(0.011614277958869934) - rollout_corr/log_ppl_diff_min:np.float64(-0.011640861630439758) - rollout_corr/ppl_ratio:np.float64(1.0006417338736355) - rollout_corr/chi2_token:np.float64(0.0015386808663606644) - rollout_corr/chi2_seq:np.float64(0.6557973187882453) - actor/pg_loss:np.float64(8.236418943852186e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0006260939494495688) - actor/ppo_kl:np.float64(0.00012735309167744013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2651471309363842) - perf/mfu/actor:np.float64(0.07338363198697957) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.38971328735352) - actor/lr:np.float64(1e-06) - training/global_step:29 - training/epoch:1 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009649740532040596 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009649740532040596 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:532.41796875 - response_length/max:2748.0 - response_length/min:73.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:532.41796875 - response_length_non_aborted/max:2748.0 - response_length_non_aborted/min:73.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.53125 - prompt_length/max:375.0 - prompt_length/min:164.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.792375981807709e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8251552730798721) - timing_s/agent_loop/generate_sequences/max:np.float64(16.657433584332466) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.999644991774403) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.657433584332466) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:265 - timing_s/agent_loop/slowest/response_length:2748 - timing_s/gen:22.55767191387713 - timing_s/reward:0.06376934424042702 - timing_s/old_log_prob:2.4361399989575148 - timing_s/adv:0.1466422975063324 - timing_s/update_actor:10.275281056761742 - timing_s/step:35.56420364230871 - timing_s/stop_profile:0.00011993944644927979 - timing_per_token_ms/update_actor:0.050427609805321585 - timing_per_token_ms/gen:0.16550137502019185 - timing_per_token_ms/adv:0.000719670879925857 - perf/total_num_tokens:203763 - perf/time_per_step:35.56420364230871 - perf/throughput:716.1801022222058 (TaskRunner pid=2049544) Training Progress: 29%|██▉ | 29/100 [19:45<40:54, 34.57s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 30} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. The formula for the electric field due to a point charge is: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} = \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_20/actor (TaskRunner pid=2049544) step:30 - global_seqlen/min:16681 - global_seqlen/max:26264 - global_seqlen/minmax_diff:9583 - global_seqlen/balanced_min:22916 - global_seqlen/balanced_max:22938 - global_seqlen/mean:22931.25 - actor/entropy:0.1647506058216095 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3170890808105469 - training/rollout_probs_diff_mean:0.0035670390352606773 - training/rollout_probs_diff_std:0.01208495907485485 - training/rollout_actor_probs_pearson_corr:0.9981399178504944 - rollout_corr/rollout_is_mean:0.9999049305915833 - rollout_corr/rollout_is_ratio_fraction_high:8.544525371689815e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.00011962335702264681 - rollout_corr/rollout_is_max:2.2645792961120605 - rollout_corr/rollout_is_min:0.1764460951089859 - rollout_corr/rollout_is_std:0.03402237966656685 - rollout_corr/rollout_is_eff_sample_size:0.9988435782394575 - rollout_corr/rollout_is_seq_mean:0.9999919533729553 - rollout_corr/rollout_is_seq_std:0.0021591568365693092 - rollout_corr/rollout_is_seq_max:1.0069876909255981 - rollout_corr/rollout_is_seq_min:0.9930654764175415 - rollout_corr/rollout_is_seq_max_deviation:0.0069876909255981445 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1598813771270216) - rollout_corr/training_log_ppl:np.float64(0.14310977509740042) - rollout_corr/kl:np.float64(0.0006737301786081673) - rollout_corr/k3_kl:np.float64(0.0006828237883240718) - rollout_corr/rollout_ppl:np.float64(1.1591023579239845) - rollout_corr/rollout_log_ppl:np.float64(0.14243604317016434) - rollout_corr/log_ppl_diff:np.float64(0.0006737319272360764) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017324861764791422) - rollout_corr/log_ppl_diff_max:np.float64(0.0082569420337677) - rollout_corr/log_ppl_diff_min:np.float64(-0.005524754524230957) - rollout_corr/ppl_ratio:np.float64(1.000676429597661) - rollout_corr/chi2_token:np.float64(0.0013968497514724731) - rollout_corr/chi2_seq:np.float64(0.42728084372356534) - actor/pg_loss:np.float64(2.553162630647421e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004172229162122676) - actor/ppo_kl:np.float64(2.4439394231023925e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.29012734070420265) - perf/mfu/actor:np.float64(0.06612954074679292) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.5312614440918) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63125) - val-aux/sciknoweval/reward/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09227713650829415) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.81775) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04889979668850037) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6632125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06499777900917662) - val-aux/sciknoweval/score/mean@16:np.float64(0.63125) - val-aux/sciknoweval/score/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09227713650829415) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.81775) - val-aux/sciknoweval/score/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04889979668850037) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6632125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06499777900917662) - val-core/sciknoweval/acc/mean@16:np.float64(0.63125) - val-aux/sciknoweval/acc/std@16:np.float64(0.16827281213257073) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.69945) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1358224817804255) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5622375) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14543653725309813) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.630625) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1674322657572851) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7523875) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09759004740494832) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5039875) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.11512292056447274) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.648175) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13191413368892382) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.7910625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06526407309863623) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45775000000000005) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08015696464235675) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6597000000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09227713650829415) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.81775) - val-core/sciknoweval/acc/best@16/std:np.float64(0.039141593967153915) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.42647500000000005) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04889979668850037) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6632125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06499777900917662) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98359375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.04653599667599353) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.997425) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.01734892838049699) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9696374999999999) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.05990914656773903) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9836) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.046352514622771394) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.999875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.00190372690402996) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9472625000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.0713750183504594) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9930749999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.029627882526401583) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9146000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.07555851957352466) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9977499999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.014792280126255261) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8794875000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06529402915117104) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9997250000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0034755281407842023) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:30 - training/epoch:1 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012676871381700039 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012676871381700039 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:457.1640625 - response_length/max:1567.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:457.1640625 - response_length_non_aborted/max:1567.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.4375 - prompt_length/max:330.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010706298053264618 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7780425660312176) - timing_s/agent_loop/generate_sequences/max:np.float64(10.887974064797163) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.538448243663879) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.887974064797163) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:303 - timing_s/agent_loop/slowest/response_length:1567 - timing_s/gen:16.679681358858943 - timing_s/reward:0.06917422637343407 - timing_s/old_log_prob:2.485142918303609 - timing_s/adv:0.12575341202318668 - timing_s/update_actor:10.15482790954411 - timing_s/step:29.608009880408645 - timing_s/testing:103.1428439784795 - timing_s/save_checkpoint:6.939728271216154 - timing_s/stop_profile:0.0001257825642824173 - timing_per_token_ms/update_actor:0.05535474466908755 - timing_per_token_ms/gen:0.14251996307789996 - timing_per_token_ms/adv:0.0006854914800936859 - perf/total_num_tokens:183450 - perf/time_per_step:29.608009880408645 - perf/throughput:774.4948104456491 (TaskRunner pid=2049544) Training Progress: 30%|███ | 30/100 [22:05<1:17:09, 66.13s/it] (TaskRunner pid=2049544) step:31 - global_seqlen/min:19206 - global_seqlen/max:40437 - global_seqlen/minmax_diff:21231 - global_seqlen/balanced_min:26988 - global_seqlen/balanced_max:27006 - global_seqlen/mean:26994.75 - actor/entropy:0.15669088065624237 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2584149241447449 - training/rollout_probs_diff_mean:0.003061925061047077 - training/rollout_probs_diff_std:0.010589882731437683 - training/rollout_actor_probs_pearson_corr:0.9985439777374268 - rollout_corr/rollout_is_mean:1.0000404119491577 - rollout_corr/rollout_is_ratio_fraction_high:1.3810819837090094e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.7621639674180187e-05 - rollout_corr/rollout_is_max:2.6365303993225098 - rollout_corr/rollout_is_min:0.22984786331653595 - rollout_corr/rollout_is_std:0.030734580010175705 - rollout_corr/rollout_is_eff_sample_size:0.9990563960283122 - rollout_corr/rollout_is_seq_mean:1.0000174045562744 - rollout_corr/rollout_is_seq_std:0.0015726868296042085 - rollout_corr/rollout_is_seq_max:1.0063960552215576 - rollout_corr/rollout_is_seq_min:0.9952640533447266 - rollout_corr/rollout_is_seq_max_deviation:0.006396055221557617 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1623710673302412) - rollout_corr/training_log_ppl:np.float64(0.14397141717927298) - rollout_corr/kl:np.float64(0.0004526557635822215) - rollout_corr/k3_kl:np.float64(0.0005084346583146271) - rollout_corr/rollout_ppl:np.float64(1.1618167613632977) - rollout_corr/rollout_log_ppl:np.float64(0.1435187609968125) - rollout_corr/log_ppl_diff:np.float64(0.0004526561824604869) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012202383077237755) - rollout_corr/log_ppl_diff_max:np.float64(0.005235195159912109) - rollout_corr/log_ppl_diff_min:np.float64(-0.004862427711486816) - rollout_corr/ppl_ratio:np.float64(1.0004539461806417) - rollout_corr/chi2_token:np.float64(0.0011370060965418816) - rollout_corr/chi2_seq:np.float64(0.43574777944013476) - actor/pg_loss:np.float64(1.602328848093748e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027079670348939544) - actor/ppo_kl:np.float64(-1.6756775252702028e-05) - actor/pg_clipfrac_lower:np.float64(2.4157391180779086e-06) - actor/grad_norm:np.float64(0.20888379216194153) - perf/mfu/actor:np.float64(0.07792531348505069) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.11697006225586) - actor/lr:np.float64(1e-06) - training/global_step:31 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006074171047657728 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.006074171047657728 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:565.6796875 - response_length/max:1784.0 - response_length/min:147.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:565.6796875 - response_length_non_aborted/max:1784.0 - response_length_non_aborted/min:147.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.90625 - prompt_length/max:364.0 - prompt_length/min:188.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.553879499435425e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7227725125849247) - timing_s/agent_loop/generate_sequences/max:np.float64(13.10448818653822) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.673034100342193) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.10448818653822) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:284 - timing_s/agent_loop/slowest/response_length:1784 - timing_s/gen:18.842551900073886 - timing_s/reward:0.07317895255982876 - timing_s/old_log_prob:2.6255607213824987 - timing_s/adv:0.14376060664653778 - timing_s/update_actor:10.008040392771363 - timing_s/step:31.798675939440727 - timing_s/stop_profile:0.00011486746370792389 - timing_per_token_ms/update_actor:0.0463425313846737 - timing_per_token_ms/gen:0.13011554062503547 - timing_per_token_ms/adv:0.000665687803399447 - perf/total_num_tokens:215958 - perf/time_per_step:31.798675939440727 - perf/throughput:848.9268563071744 (TaskRunner pid=2049544) Training Progress: 31%|███ | 31/100 [22:36<1:04:13, 55.85s/it] (TaskRunner pid=2049544) step:32 - global_seqlen/min:15472 - global_seqlen/max:26624 - global_seqlen/minmax_diff:11152 - global_seqlen/balanced_min:21157 - global_seqlen/balanced_max:21170 - global_seqlen/mean:21164.125 - actor/entropy:0.16215530037879944 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35835716128349304 - training/rollout_probs_diff_mean:0.0033622696064412594 - training/rollout_probs_diff_std:0.011649704538285732 - training/rollout_actor_probs_pearson_corr:0.9983426928520203 - rollout_corr/rollout_is_mean:0.9998896718025208 - rollout_corr/rollout_is_ratio_fraction_high:1.9218758097849786e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.765627429354936e-05 - rollout_corr/rollout_is_max:2.1169991493225098 - rollout_corr/rollout_is_min:0.016281770542263985 - rollout_corr/rollout_is_std:0.033417847007513046 - rollout_corr/rollout_is_eff_sample_size:0.9988841362904098 - rollout_corr/rollout_is_seq_mean:0.9999381303787231 - rollout_corr/rollout_is_seq_std:0.00247986800968647 - rollout_corr/rollout_is_seq_max:1.011742353439331 - rollout_corr/rollout_is_seq_min:0.9914096593856812 - rollout_corr/rollout_is_seq_max_deviation:0.011742353439331055 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1614093114621937) - rollout_corr/training_log_ppl:np.float64(0.1426902024759329) - rollout_corr/kl:np.float64(0.0009178641532645582) - rollout_corr/k3_kl:np.float64(0.0007639187314083529) - rollout_corr/rollout_ppl:np.float64(1.1602927432395518) - rollout_corr/rollout_log_ppl:np.float64(0.1417723370686872) - rollout_corr/log_ppl_diff:np.float64(0.0009178654072456993) - rollout_corr/log_ppl_abs_diff:np.float64(0.0020935917491442524) - rollout_corr/log_ppl_diff_max:np.float64(0.03533336520195007) - rollout_corr/log_ppl_diff_min:np.float64(-0.009082436561584473) - rollout_corr/ppl_ratio:np.float64(1.000924628926441) - rollout_corr/chi2_token:np.float64(0.00115761230699718) - rollout_corr/chi2_seq:np.float64(0.4135602100286633) - actor/pg_loss:np.float64(-7.768673822283745e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003810818443525932) - actor/ppo_kl:np.float64(0.00014974098553555137) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.26316603645682335) - perf/mfu/actor:np.float64(0.0603751772224912) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.58000946044922) - actor/lr:np.float64(1e-06) - training/global_step:32 - training/epoch:1 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003006534418091178 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003006534418091178 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:406.50390625 - response_length/max:1261.0 - response_length/min:50.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:406.50390625 - response_length_non_aborted/max:1261.0 - response_length_non_aborted/min:50.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.875 - prompt_length/max:375.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013849511742591858 - timing_s/agent_loop/generate_sequences/min:np.float64(0.584519986063242) - timing_s/agent_loop/generate_sequences/max:np.float64(9.110312020406127) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.030619656012277) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(9.110312020406127) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:1261 - timing_s/gen:14.926933655515313 - timing_s/reward:0.05845559947192669 - timing_s/old_log_prob:2.34299498423934 - timing_s/adv:0.13051477633416653 - timing_s/update_actor:10.037976827472448 - timing_s/step:27.581528935581446 - timing_s/stop_profile:0.00012824125587940216 - timing_per_token_ms/update_actor:0.05928650976282063 - timing_per_token_ms/gen:0.14343855912665462 - timing_per_token_ms/adv:0.000770849115745197 - perf/total_num_tokens:169313 - perf/time_per_step:27.581528935581446 - perf/throughput:767.3296520084244 (TaskRunner pid=2049544) Training Progress: 32%|███▏ | 32/100 [23:04<53:42, 47.39s/it] (TaskRunner pid=2049544) step:33 - global_seqlen/min:21009 - global_seqlen/max:35793 - global_seqlen/minmax_diff:14784 - global_seqlen/balanced_min:26523 - global_seqlen/balanced_max:26619 - global_seqlen/mean:26546.375 - actor/entropy:0.1607956439256668 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4141917824745178 - training/rollout_probs_diff_mean:0.0031411326490342617 - training/rollout_probs_diff_std:0.01075077150017023 - training/rollout_actor_probs_pearson_corr:0.998579204082489 - rollout_corr/rollout_is_mean:1.0000613927841187 - rollout_corr/rollout_is_ratio_fraction_high:2.17348788282834e-05 - rollout_corr/rollout_is_ratio_fraction_low:5.0714716053334996e-05 - rollout_corr/rollout_is_max:5.24385929107666 - rollout_corr/rollout_is_min:0.3405860662460327 - rollout_corr/rollout_is_std:0.0315140075981617 - rollout_corr/rollout_is_eff_sample_size:0.9990078527425291 - rollout_corr/rollout_is_seq_mean:1.0000804662704468 - rollout_corr/rollout_is_seq_std:0.0017564821755513549 - rollout_corr/rollout_is_seq_max:1.006698489189148 - rollout_corr/rollout_is_seq_min:0.995316743850708 - rollout_corr/rollout_is_seq_max_deviation:0.006698489189147949 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.176677621435374) - rollout_corr/training_log_ppl:np.float64(0.15545551202376373) - rollout_corr/kl:np.float64(0.000548688222613336) - rollout_corr/k3_kl:np.float64(0.0006223271676191189) - rollout_corr/rollout_ppl:np.float64(1.1760094286873937) - rollout_corr/rollout_log_ppl:np.float64(0.1549068232634454) - rollout_corr/log_ppl_diff:np.float64(0.0005486887603183277) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014138846672722138) - rollout_corr/log_ppl_diff_max:np.float64(0.008264932781457901) - rollout_corr/log_ppl_diff_min:np.float64(-0.005941852927207947) - rollout_corr/ppl_ratio:np.float64(1.0005506034940481) - rollout_corr/chi2_token:np.float64(0.0014625547919422388) - rollout_corr/chi2_seq:np.float64(0.45368791883811355) - actor/pg_loss:np.float64(5.0174305215477943e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003640777423470354) - actor/ppo_kl:np.float64(6.6643967455704e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2803301140666008) - perf/mfu/actor:np.float64(0.07602373904107226) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.56497955322266) - actor/lr:np.float64(1e-06) - training/global_step:33 - training/epoch:1 - critic/score/mean:0.6953125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6953125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006237910129129887 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006237910129129887 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:539.16796875 - response_length/max:2340.0 - response_length/min:114.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:539.16796875 - response_length_non_aborted/max:2340.0 - response_length_non_aborted/min:114.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:290.40625 - prompt_length/max:437.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011204741895198822 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2724983282387257) - timing_s/agent_loop/generate_sequences/max:np.float64(14.548386441543698) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.296186965904781) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.548386441543698) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:2340 - timing_s/gen:20.489616226404905 - timing_s/reward:0.06211276724934578 - timing_s/old_log_prob:2.5394117888063192 - timing_s/adv:0.15707547590136528 - timing_s/update_actor:10.15939660742879 - timing_s/step:33.50294020399451 - timing_s/stop_profile:0.00015977025032043457 - timing_per_token_ms/update_actor:0.04783796567058962 - timing_per_token_ms/gen:0.14844643603356522 - timing_per_token_ms/adv:0.0007396277076501277 - perf/total_num_tokens:212371 - perf/time_per_step:33.50294020399451 - perf/throughput:792.3595612314322 (TaskRunner pid=2049544) Training Progress: 33%|███▎ | 33/100 [23:38<48:17, 43.24s/it] (TaskRunner pid=2049544) step:34 - global_seqlen/min:19210 - global_seqlen/max:32362 - global_seqlen/minmax_diff:13152 - global_seqlen/balanced_min:24672 - global_seqlen/balanced_max:24722 - global_seqlen/mean:24706.625 - actor/entropy:0.1634763479232788 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967529773712158 - training/rollout_probs_diff_mean:0.003025032114237547 - training/rollout_probs_diff_std:0.010739969089627266 - training/rollout_actor_probs_pearson_corr:0.998622715473175 - rollout_corr/rollout_is_mean:1.0000046491622925 - rollout_corr/rollout_is_ratio_fraction_high:2.2830181478639133e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.84905462549068e-05 - rollout_corr/rollout_is_max:3.96623158454895 - rollout_corr/rollout_is_min:0.17613501846790314 - rollout_corr/rollout_is_std:0.03082752786576748 - rollout_corr/rollout_is_eff_sample_size:0.9990505658256099 - rollout_corr/rollout_is_seq_mean:1.0001435279846191 - rollout_corr/rollout_is_seq_std:0.001950009143911302 - rollout_corr/rollout_is_seq_max:1.0084269046783447 - rollout_corr/rollout_is_seq_min:0.9940277338027954 - rollout_corr/rollout_is_seq_max_deviation:0.008426904678344727 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1493624411523342) - rollout_corr/training_log_ppl:np.float64(0.13525041530374438) - rollout_corr/kl:np.float64(0.0005815880843990096) - rollout_corr/k3_kl:np.float64(0.0006954628924091821) - rollout_corr/rollout_ppl:np.float64(1.1486861389130354) - rollout_corr/rollout_log_ppl:np.float64(0.13466882749344222) - rollout_corr/log_ppl_diff:np.float64(0.0005815878103021532) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016140752850333229) - rollout_corr/log_ppl_diff_max:np.float64(0.00731133297085762) - rollout_corr/log_ppl_diff_min:np.float64(-0.007330030202865601) - rollout_corr/ppl_ratio:np.float64(1.0005840898957103) - rollout_corr/chi2_token:np.float64(0.0018483358435332775) - rollout_corr/chi2_seq:np.float64(0.8693230981007218) - actor/pg_loss:np.float64(-7.774424739181995e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0004309596388338832) - actor/ppo_kl:np.float64(0.00016126566817398214) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.24917255342006683) - perf/mfu/actor:np.float64(0.07059737130291072) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.48730087280273) - actor/lr:np.float64(1e-06) - training/global_step:34 - training/epoch:1 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.002149841981008649 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.002149841981008649 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:513.30078125 - response_length/max:2220.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:513.30078125 - response_length_non_aborted/max:2220.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.78125 - prompt_length/max:382.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001704581081867218 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9642345681786537) - timing_s/agent_loop/generate_sequences/max:np.float64(14.358597544953227) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.930665408472123) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.358597544953227) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:2220 - timing_s/gen:20.079416058957577 - timing_s/reward:0.055915795266628265 - timing_s/old_log_prob:2.5313415098935366 - timing_s/adv:0.14141708984971046 - timing_s/update_actor:10.167155608534813 - timing_s/step:33.08425388485193 - timing_s/stop_profile:0.0001313425600528717 - timing_per_token_ms/update_actor:0.05143941963205625 - timing_per_token_ms/gen:0.15280557101295672 - timing_per_token_ms/adv:0.000715481626131202 - perf/total_num_tokens:197653 - perf/time_per_step:33.08425388485193 - perf/throughput:746.7789688106661 (TaskRunner pid=2049544) Training Progress: 34%|███▍ | 34/100 [24:11<44:13, 40.21s/it] (TaskRunner pid=2049544) step:35 - global_seqlen/min:18610 - global_seqlen/max:31076 - global_seqlen/minmax_diff:12466 - global_seqlen/balanced_min:24486 - global_seqlen/balanced_max:24516 - global_seqlen/mean:24505.375 - actor/entropy:0.18308240175247192 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6351485848426819 - training/rollout_probs_diff_mean:0.003324724268168211 - training/rollout_probs_diff_std:0.011287217028439045 - training/rollout_actor_probs_pearson_corr:0.998609185218811 - rollout_corr/rollout_is_mean:0.9998504519462585 - rollout_corr/rollout_is_ratio_fraction_high:1.5409863408422098e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.852466034004465e-05 - rollout_corr/rollout_is_max:3.141425609588623 - rollout_corr/rollout_is_min:0.22313041985034943 - rollout_corr/rollout_is_std:0.03237971290946007 - rollout_corr/rollout_is_eff_sample_size:0.9989524144661357 - rollout_corr/rollout_is_seq_mean:0.999904215335846 - rollout_corr/rollout_is_seq_std:0.002002937952056527 - rollout_corr/rollout_is_seq_max:1.0084916353225708 - rollout_corr/rollout_is_seq_min:0.9936511516571045 - rollout_corr/rollout_is_seq_max_deviation:0.0084916353225708 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1920651025138795) - rollout_corr/training_log_ppl:np.float64(0.16792560920293909) - rollout_corr/kl:np.float64(0.0008154499521539904) - rollout_corr/k3_kl:np.float64(0.0007078099062596266) - rollout_corr/rollout_ppl:np.float64(1.1910648443736136) - rollout_corr/rollout_log_ppl:np.float64(0.1671101573156193) - rollout_corr/log_ppl_diff:np.float64(0.0008154518873197958) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017030747985700145) - rollout_corr/log_ppl_diff_max:np.float64(0.010384440422058105) - rollout_corr/log_ppl_diff_min:np.float64(-0.007758818566799164) - rollout_corr/ppl_ratio:np.float64(1.0008183948229998) - rollout_corr/chi2_token:np.float64(0.0012777894735336304) - rollout_corr/chi2_seq:np.float64(1.3958141808398068) - actor/pg_loss:np.float64(1.1038966476917267e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003179428899784398) - actor/ppo_kl:np.float64(0.00012983981464032013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2326635681092739) - perf/mfu/actor:np.float64(0.07068896275098592) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.57692337036133) - actor/lr:np.float64(1e-06) - training/global_step:35 - training/epoch:1 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.004130806773900986 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.004130806773900986 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:506.98046875 - response_length/max:1617.0 - response_length/min:86.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:506.98046875 - response_length_non_aborted/max:1617.0 - response_length_non_aborted/min:86.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.8125 - prompt_length/max:365.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.149631321430206e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9240005984902382) - timing_s/agent_loop/generate_sequences/max:np.float64(11.936645423993468) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.090721758017025) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(11.936645423993468) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:1617 - timing_s/gen:17.894150488078594 - timing_s/reward:0.06287588179111481 - timing_s/old_log_prob:2.4778820667415857 - timing_s/adv:0.1439337357878685 - timing_s/update_actor:10.013244837522507 - timing_s/step:30.680053589865565 - timing_s/stop_profile:0.00010856986045837402 - timing_per_token_ms/update_actor:0.05107677824519369 - timing_per_token_ms/gen:0.13787321140082284 - timing_per_token_ms/adv:0.0007341947215043052 - perf/total_num_tokens:196043 - perf/time_per_step:30.680053589865565 - perf/throughput:798.7396413184485 (TaskRunner pid=2049544) Training Progress: 35%|███▌ | 35/100 [24:41<40:28, 37.37s/it] (TaskRunner pid=2049544) step:36 - global_seqlen/min:24110 - global_seqlen/max:37995 - global_seqlen/minmax_diff:13885 - global_seqlen/balanced_min:31143 - global_seqlen/balanced_max:31201 - global_seqlen/mean:31181.25 - actor/entropy:0.2063475251197815 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35195159912109375 - training/rollout_probs_diff_mean:0.003259577788412571 - training/rollout_probs_diff_std:0.010389592498540878 - training/rollout_actor_probs_pearson_corr:0.9988572597503662 - rollout_corr/rollout_is_mean:0.9999135732650757 - rollout_corr/rollout_is_ratio_fraction_high:5.54268399355351e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.217073597421404e-05 - rollout_corr/rollout_is_max:5.167435169219971 - rollout_corr/rollout_is_min:0.08687802404165268 - rollout_corr/rollout_is_std:0.03102603740990162 - rollout_corr/rollout_is_eff_sample_size:0.9990380727632393 - rollout_corr/rollout_is_seq_mean:0.9999022483825684 - rollout_corr/rollout_is_seq_std:0.0015286069829016924 - rollout_corr/rollout_is_seq_max:1.0051732063293457 - rollout_corr/rollout_is_seq_min:0.9951693415641785 - rollout_corr/rollout_is_seq_max_deviation:0.005173206329345703 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2262746798805892) - rollout_corr/training_log_ppl:np.float64(0.19696971234225202) - rollout_corr/kl:np.float64(0.0006086873604012055) - rollout_corr/k3_kl:np.float64(0.0005889998031136656) - rollout_corr/rollout_ppl:np.float64(1.2254916024394333) - rollout_corr/rollout_log_ppl:np.float64(0.19636102448566817) - rollout_corr/log_ppl_diff:np.float64(0.000608687856583856) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012519802548922598) - rollout_corr/log_ppl_diff_max:np.float64(0.006760932505130768) - rollout_corr/log_ppl_diff_min:np.float64(-0.005945771932601929) - rollout_corr/ppl_ratio:np.float64(1.0006101776380092) - rollout_corr/chi2_token:np.float64(0.001163150416687131) - rollout_corr/chi2_seq:np.float64(0.9006679092999548) - actor/pg_loss:np.float64(6.59312354400754e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041949365038362885) - actor/ppo_kl:np.float64(-4.479620235997572e-05) - actor/pg_clipfrac_lower:np.float64(3.426535158723709e-06) - actor/grad_norm:np.float64(0.21542762964963913) - perf/mfu/actor:np.float64(0.08866368508784078) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.68338394165039) - actor/lr:np.float64(1e-06) - training/global_step:36 - training/epoch:1 - critic/score/mean:0.56640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.56640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.03192170336842537 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.03192170336842537 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:704.7578125 - response_length/max:2399.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:704.7578125 - response_length_non_aborted/max:2399.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.65625 - prompt_length/max:375.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.613484144210815e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1882787812501192) - timing_s/agent_loop/generate_sequences/max:np.float64(16.427959153428674) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.1280951405206) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.427959153428674) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:263 - timing_s/agent_loop/slowest/response_length:2399 - timing_s/gen:22.231742713600397 - timing_s/reward:0.06625362113118172 - timing_s/old_log_prob:2.5154502913355827 - timing_s/adv:0.1720115952193737 - timing_s/update_actor:10.132180897518992 - timing_s/step:35.22234913893044 - timing_s/stop_profile:0.00012091919779777527 - timing_per_token_ms/update_actor:0.04061808337349766 - timing_per_token_ms/gen:0.12322352932412729 - timing_per_token_ms/adv:0.00068956342040238 - perf/total_num_tokens:249450 - perf/time_per_step:35.22234913893044 - perf/throughput:885.2688921175928 (TaskRunner pid=2049544) Training Progress: 36%|███▌ | 36/100 [25:17<39:11, 36.74s/it] (TaskRunner pid=2049544) step:37 - global_seqlen/min:20543 - global_seqlen/max:33018 - global_seqlen/minmax_diff:12475 - global_seqlen/balanced_min:26755 - global_seqlen/balanced_max:26801 - global_seqlen/mean:26782.75 - actor/entropy:0.1873542070388794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535153567790985 - training/rollout_probs_diff_mean:0.003138389438390732 - training/rollout_probs_diff_std:0.010506313294172287 - training/rollout_actor_probs_pearson_corr:0.9987807273864746 - rollout_corr/rollout_is_mean:0.9999997019767761 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.388498225831427e-05 - rollout_corr/rollout_is_max:1.984167218208313 - rollout_corr/rollout_is_min:0.38167211413383484 - rollout_corr/rollout_is_std:0.031033720821142197 - rollout_corr/rollout_is_eff_sample_size:0.9990378348031789 - rollout_corr/rollout_is_seq_mean:0.9999757409095764 - rollout_corr/rollout_is_seq_std:0.0020235180854797363 - rollout_corr/rollout_is_seq_max:1.007972240447998 - rollout_corr/rollout_is_seq_min:0.991079568862915 - rollout_corr/rollout_is_seq_max_deviation:0.008920431137084961 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.17386184213683) - rollout_corr/training_log_ppl:np.float64(0.15399431145488052) - rollout_corr/kl:np.float64(0.000585384931017785) - rollout_corr/k3_kl:np.float64(0.0005648392770467581) - rollout_corr/rollout_ppl:np.float64(1.1731591601856053) - rollout_corr/rollout_log_ppl:np.float64(0.15340892578387866) - rollout_corr/log_ppl_diff:np.float64(0.000585385671001859) - rollout_corr/log_ppl_abs_diff:np.float64(0.0015394459333037958) - rollout_corr/log_ppl_diff_max:np.float64(0.009715557098388672) - rollout_corr/log_ppl_diff_min:np.float64(-0.006565690040588379) - rollout_corr/ppl_ratio:np.float64(1.0005877201911062) - rollout_corr/chi2_token:np.float64(0.0010866590309888124) - rollout_corr/chi2_seq:np.float64(1.2919703791849315) - actor/pg_loss:np.float64(-7.207388989627361e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027252964378021716) - actor/ppo_kl:np.float64(2.565641079144143e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2241645734757185) - perf/mfu/actor:np.float64(0.07756524385091812) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.62493515014648) - actor/lr:np.float64(1e-06) - training/global_step:37 - training/epoch:1 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012340063229203224 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012340063229203224 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:576.3984375 - response_length/max:2563.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:576.3984375 - response_length_non_aborted/max:2563.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:260.5625 - prompt_length/max:350.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.650814950466156e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7121520359069109) - timing_s/agent_loop/generate_sequences/max:np.float64(16.06510110758245) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.399860835161235) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.06510110758245) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2563 - timing_s/gen:21.823639852926135 - timing_s/reward:0.07287336513400078 - timing_s/old_log_prob:2.61249627918005 - timing_s/adv:0.16705027595162392 - timing_s/update_actor:10.120526600629091 - timing_s/step:34.892802860587835 - timing_s/stop_profile:0.00011398270726203918 - timing_per_token_ms/update_actor:0.04723435140449119 - timing_per_token_ms/gen:0.14789872357260286 - timing_per_token_ms/adv:0.0007796542361763818 - perf/total_num_tokens:214262 - perf/time_per_step:34.892802860587835 - perf/throughput:767.5723302312205 (TaskRunner pid=2049544) Training Progress: 37%|███▋ | 37/100 [25:52<38:00, 36.20s/it] (TaskRunner pid=2049544) step:38 - global_seqlen/min:20896 - global_seqlen/max:36221 - global_seqlen/minmax_diff:15325 - global_seqlen/balanced_min:26629 - global_seqlen/balanced_max:26693 - global_seqlen/mean:26673.0 - actor/entropy:0.17777901887893677 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311151742935181 - training/rollout_probs_diff_mean:0.003014863934367895 - training/rollout_probs_diff_std:0.01037665270268917 - training/rollout_actor_probs_pearson_corr:0.998752772808075 - rollout_corr/rollout_is_mean:1.0000866651535034 - rollout_corr/rollout_is_ratio_fraction_high:2.75694746960653e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.446184564381838e-05 - rollout_corr/rollout_is_max:4.722990036010742 - rollout_corr/rollout_is_min:0.217980295419693 - rollout_corr/rollout_is_std:0.03060242161154747 - rollout_corr/rollout_is_eff_sample_size:0.9990646060209646 - rollout_corr/rollout_is_seq_mean:1.0000120401382446 - rollout_corr/rollout_is_seq_std:0.0017710411921143532 - rollout_corr/rollout_is_seq_max:1.005569338798523 - rollout_corr/rollout_is_seq_min:0.9887515902519226 - rollout_corr/rollout_is_seq_max_deviation:0.011248409748077393 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1705434750765562) - rollout_corr/training_log_ppl:np.float64(0.15039988774515223) - rollout_corr/kl:np.float64(0.0003758689842143781) - rollout_corr/k3_kl:np.float64(0.0005343400661388387) - rollout_corr/rollout_ppl:np.float64(1.170116110239178) - rollout_corr/rollout_log_ppl:np.float64(0.1500240175955696) - rollout_corr/log_ppl_diff:np.float64(0.00037587014958262444) - rollout_corr/log_ppl_abs_diff:np.float64(0.00122680542699527) - rollout_corr/log_ppl_diff_max:np.float64(0.012495182454586029) - rollout_corr/log_ppl_diff_min:np.float64(-0.005860418081283569) - rollout_corr/ppl_ratio:np.float64(1.000377441989258) - rollout_corr/chi2_token:np.float64(0.0015310256276279688) - rollout_corr/chi2_seq:np.float64(0.8941571062896401) - actor/pg_loss:np.float64(7.285538595169783e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002832129719081422) - actor/ppo_kl:np.float64(-9.715641379060003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20185822248458862) - perf/mfu/actor:np.float64(0.07681988024557596) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.53835678100586) - actor/lr:np.float64(1e-06) - training/global_step:38 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0021598960738629103 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0021598960738629103 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:566.75 - response_length/max:2166.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:566.75 - response_length_non_aborted/max:2166.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.78125 - prompt_length/max:328.0 - prompt_length/min:180.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.976459503173828e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1030525546520948) - timing_s/agent_loop/generate_sequences/max:np.float64(14.58826527185738) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.473840224818559) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.58826527185738) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:328 - timing_s/agent_loop/slowest/response_length:2166 - timing_s/gen:20.538045482710004 - timing_s/reward:0.06479933671653271 - timing_s/old_log_prob:2.464496372267604 - timing_s/adv:0.16802607849240303 - timing_s/update_actor:10.160526245832443 - timing_s/step:33.478492356836796 - timing_s/stop_profile:0.00017846189439296722 - timing_per_token_ms/update_actor:0.04761615793982887 - timing_per_token_ms/gen:0.1415557832674653 - timing_per_token_ms/adv:0.0007874352270667109 - perf/total_num_tokens:213384 - perf/time_per_step:33.478492356836796 - perf/throughput:796.720465058606 (TaskRunner pid=2049544) Training Progress: 38%|███▊ | 38/100 [26:25<36:34, 35.40s/it] (TaskRunner pid=2049544) step:39 - global_seqlen/min:15255 - global_seqlen/max:27450 - global_seqlen/minmax_diff:12195 - global_seqlen/balanced_min:22238 - global_seqlen/balanced_max:22271 - global_seqlen/mean:22258.375 - actor/entropy:0.1585860699415207 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.403981477022171 - training/rollout_probs_diff_mean:0.003201692830771208 - training/rollout_probs_diff_std:0.011194097809493542 - training/rollout_actor_probs_pearson_corr:0.9984230995178223 - rollout_corr/rollout_is_mean:1.0000526905059814 - rollout_corr/rollout_is_ratio_fraction_high:4.465760866878554e-05 - rollout_corr/rollout_is_ratio_fraction_low:6.252065213629976e-05 - rollout_corr/rollout_is_max:3.0618841648101807 - rollout_corr/rollout_is_min:0.25154978036880493 - rollout_corr/rollout_is_std:0.032896533608436584 - rollout_corr/rollout_is_eff_sample_size:0.9989191068729061 - rollout_corr/rollout_is_seq_mean:1.0000567436218262 - rollout_corr/rollout_is_seq_std:0.0022173086181282997 - rollout_corr/rollout_is_seq_max:1.0083491802215576 - rollout_corr/rollout_is_seq_min:0.9902656674385071 - rollout_corr/rollout_is_seq_max_deviation:0.00973433256149292 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1577396178618073) - rollout_corr/training_log_ppl:np.float64(0.13937123074720148) - rollout_corr/kl:np.float64(0.00047606502702546294) - rollout_corr/k3_kl:np.float64(0.0006253823398196801) - rollout_corr/rollout_ppl:np.float64(1.157200226560235) - rollout_corr/rollout_log_ppl:np.float64(0.1388951658955193) - rollout_corr/log_ppl_diff:np.float64(0.0004760648516821675) - rollout_corr/log_ppl_abs_diff:np.float64(0.0017094570139306597) - rollout_corr/log_ppl_diff_max:np.float64(0.008976280689239502) - rollout_corr/log_ppl_diff_min:np.float64(-0.007652416825294495) - rollout_corr/ppl_ratio:np.float64(1.0004788676742464) - rollout_corr/chi2_token:np.float64(0.001593738328665495) - rollout_corr/chi2_seq:np.float64(1.153579653473571) - actor/pg_loss:np.float64(-4.086887929588556e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021084232048451668) - actor/ppo_kl:np.float64(-5.276580297675082e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22345420345664024) - perf/mfu/actor:np.float64(0.06472844108846296) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.6031494140625) - actor/lr:np.float64(1e-06) - training/global_step:39 - training/epoch:1 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0039901575073599815 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0039901575073599815 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:437.35546875 - response_length/max:1579.0 - response_length/min:66.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:437.35546875 - response_length_non_aborted/max:1579.0 - response_length_non_aborted/min:66.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.21875 - prompt_length/max:326.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015556812286376953 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7400109507143497) - timing_s/agent_loop/generate_sequences/max:np.float64(10.687160091474652) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.344982730093761) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(10.687160091474652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:300 - timing_s/agent_loop/slowest/response_length:1564 - timing_s/gen:16.393727527931333 - timing_s/reward:0.060859134420752525 - timing_s/old_log_prob:2.4244355112314224 - timing_s/adv:0.15019422583281994 - timing_s/update_actor:9.942341027781367 - timing_s/step:29.05965017527342 - timing_s/stop_profile:0.00014510564506053925 - timing_per_token_ms/update_actor:0.05583483198897812 - timing_per_token_ms/gen:0.14642093841654236 - timing_per_token_ms/adv:0.0008434702995660057 - perf/total_num_tokens:178067 - perf/time_per_step:29.05965017527342 - perf/throughput:765.954678247966 (TaskRunner pid=2049544) Training Progress: 39%|███▉ | 39/100 [26:54<34:04, 33.52s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 40} (TaskRunner pid=2049544) (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given the electric field $ E = 2.0 \, \mathrm{N/C} $, the distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $, and the constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $. We need to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_30/actor (TaskRunner pid=2049544) step:40 - global_seqlen/min:20667 - global_seqlen/max:43425 - global_seqlen/minmax_diff:22758 - global_seqlen/balanced_min:28994 - global_seqlen/balanced_max:29096 - global_seqlen/mean:29057.375 - actor/entropy:0.16825498640537262 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.38080501556396484 - training/rollout_probs_diff_mean:0.002904911758378148 - training/rollout_probs_diff_std:0.010255875997245312 - training/rollout_actor_probs_pearson_corr:0.9987383484840393 - rollout_corr/rollout_is_mean:0.9999452829360962 - rollout_corr/rollout_is_ratio_fraction_high:1.811298898246605e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.62259779649321e-05 - rollout_corr/rollout_is_max:2.355332374572754 - rollout_corr/rollout_is_min:0.382403701543808 - rollout_corr/rollout_is_std:0.029906976968050003 - rollout_corr/rollout_is_eff_sample_size:0.9991061339931943 - rollout_corr/rollout_is_seq_mean:0.9999656081199646 - rollout_corr/rollout_is_seq_std:0.0016488394467160106 - rollout_corr/rollout_is_seq_max:1.009485125541687 - rollout_corr/rollout_is_seq_min:0.9944038391113281 - rollout_corr/rollout_is_seq_max_deviation:0.009485125541687012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.157492823433131) - rollout_corr/training_log_ppl:np.float64(0.14261198020540178) - rollout_corr/kl:np.float64(0.0005399936895650326) - rollout_corr/k3_kl:np.float64(0.0005378889725164981) - rollout_corr/rollout_ppl:np.float64(1.156866628676653) - rollout_corr/rollout_log_ppl:np.float64(0.1420719847228611) - rollout_corr/log_ppl_diff:np.float64(0.0005399954825406894) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012537419388536364) - rollout_corr/log_ppl_diff_max:np.float64(0.009929120540618896) - rollout_corr/log_ppl_diff_min:np.float64(-0.007549874484539032) - rollout_corr/ppl_ratio:np.float64(1.0005416304338723) - rollout_corr/chi2_token:np.float64(0.0010685438755899668) - rollout_corr/chi2_seq:np.float64(1.119046370498836) - actor/pg_loss:np.float64(-2.1900166757404804e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024728815151320305) - actor/ppo_kl:np.float64(-7.47417178104115e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1894925720989704) - perf/mfu/actor:np.float64(0.08261510768114952) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.61338424682617) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/reward/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.0950390140033621) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8010999999999999) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04192728514733711) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.654725) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.07283536359808326) - val-aux/sciknoweval/score/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/score/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/score/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/score/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.0950390140033621) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8010999999999999) - val-aux/sciknoweval/score/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04192728514733711) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.654725) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.07283536359808326) - val-core/sciknoweval/acc/mean@16:np.float64(0.62421875) - val-aux/sciknoweval/acc/std@16:np.float64(0.16084149793846786) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.6912874999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13357408072652655) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5557125) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.14105131014378522) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.623125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1618698771783546) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7462375) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.0922266430818953) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5021) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.10738293756009418) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6433249999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13041149154323356) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.78285) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.05523922352229523) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.45817499999999994) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07435729339376719) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.653275) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.0950390140033621) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8010999999999999) - val-core/sciknoweval/acc/best@16/std:np.float64(0.02694306919694267) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.42795000000000005) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04192728514733711) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.654725) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.07283536359808326) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9609375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.08980726114743592) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9929124999999999) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.038290550480143085) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9296875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.11086911079562345) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9614625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.08902370132140851) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9996375000000001) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0060893990521829235) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.8815249999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.12288515049099524) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9828125) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.060772813601514916) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8237249999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.11143746284761198) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9951625) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.03001908595873743) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.7747124999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0752713841460063) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99925) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.008297615319292063) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:40 - training/epoch:1 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004764470737427473 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004764470737427473 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:646.98046875 - response_length/max:3478.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.98046875 - response_length_non_aborted/max:3478.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.0625 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.821169078350067e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9843795355409384) - timing_s/agent_loop/generate_sequences/max:np.float64(20.97976647876203) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.125959783450526) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.97976647876203) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:313 - timing_s/agent_loop/slowest/response_length:3478 - timing_s/gen:26.76995438337326 - timing_s/reward:0.06942468881607056 - timing_s/old_log_prob:2.522913720458746 - timing_s/adv:0.12163343653082848 - timing_s/update_actor:10.376626841723919 - timing_s/step:39.94687524065375 - timing_s/testing:129.93085425533354 - timing_s/save_checkpoint:6.843135168775916 - timing_s/stop_profile:9.070523083209991e-05 - timing_per_token_ms/update_actor:0.044638524822544705 - timing_per_token_ms/gen:0.1616279615242277 - timing_per_token_ms/adv:0.0005232468372092648 - perf/total_num_tokens:232459 - perf/time_per_step:39.94687524065375 - perf/throughput:727.4004493454958 (TaskRunner pid=2049544) Training Progress: 40%|████ | 40/100 [29:51<1:16:29, 76.50s/it] (TaskRunner pid=2049544) step:41 - global_seqlen/min:24357 - global_seqlen/max:35343 - global_seqlen/minmax_diff:10986 - global_seqlen/balanced_min:29480 - global_seqlen/balanced_max:29507 - global_seqlen/mean:29501.125 - actor/entropy:0.1756914258003235 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44652730226516724 - training/rollout_probs_diff_mean:0.00312806130386889 - training/rollout_probs_diff_std:0.010616622865200043 - training/rollout_actor_probs_pearson_corr:0.9987709522247314 - rollout_corr/rollout_is_mean:0.9999130368232727 - rollout_corr/rollout_is_ratio_fraction_high:1.81861159944674e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.424815647827927e-05 - rollout_corr/rollout_is_max:3.465665340423584 - rollout_corr/rollout_is_min:0.21781140565872192 - rollout_corr/rollout_is_std:0.031423088163137436 - rollout_corr/rollout_is_eff_sample_size:0.9990134444982739 - rollout_corr/rollout_is_seq_mean:0.9999098777770996 - rollout_corr/rollout_is_seq_std:0.0014667019713670015 - rollout_corr/rollout_is_seq_max:1.0048595666885376 - rollout_corr/rollout_is_seq_min:0.9947068691253662 - rollout_corr/rollout_is_seq_max_deviation:0.005293130874633789 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1796441837213933) - rollout_corr/training_log_ppl:np.float64(0.16063264349941164) - rollout_corr/kl:np.float64(0.0006431586379722631) - rollout_corr/k3_kl:np.float64(0.0005324390123746525) - rollout_corr/rollout_ppl:np.float64(1.178866429720074) - rollout_corr/rollout_log_ppl:np.float64(0.15998948468040908) - rollout_corr/log_ppl_diff:np.float64(0.000643158819002565) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012713416290353052) - rollout_corr/log_ppl_diff_max:np.float64(0.007460832595825195) - rollout_corr/log_ppl_diff_min:np.float64(-0.0031778663396835327) - rollout_corr/ppl_ratio:np.float64(1.0006445937324315) - rollout_corr/chi2_token:np.float64(0.0008295027073472738) - rollout_corr/chi2_seq:np.float64(0.9563508839346468) - actor/pg_loss:np.float64(9.131035767495632e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002985205437653349) - actor/ppo_kl:np.float64(3.691702790131757e-05) - actor/pg_clipfrac_lower:np.float64(5.99118084210204e-06) - actor/grad_norm:np.float64(0.2258460782468319) - perf/mfu/actor:np.float64(0.08484757615781298) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.7967414855957) - actor/lr:np.float64(1e-06) - training/global_step:41 - training/epoch:1 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01819217950105667 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01819217950105667 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:644.37890625 - response_length/max:1915.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:644.37890625 - response_length_non_aborted/max:1915.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:361.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.878321826457977e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2621874269098043) - timing_s/agent_loop/generate_sequences/max:np.float64(14.428985564038157) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.6339426440390525) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.428985564038157) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:345 - timing_s/agent_loop/slowest/response_length:1915 - timing_s/gen:20.114043543115258 - timing_s/reward:0.06382922269403934 - timing_s/old_log_prob:2.600179996341467 - timing_s/adv:0.1322335172444582 - timing_s/update_actor:9.928239937871695 - timing_s/step:32.94103864394128 - timing_s/stop_profile:0.00011893920600414276 - timing_per_token_ms/update_actor:0.04206720903809471 - timing_per_token_ms/gen:0.12193211451867568 - timing_per_token_ms/adv:0.0005602901467505824 - perf/total_num_tokens:236009 - perf/time_per_step:32.94103864394128 - perf/throughput:895.5736131722133 (TaskRunner pid=2049544) Training Progress: 41%|████ | 41/100 [30:24<1:02:23, 63.45s/it] (TaskRunner pid=2049544) step:42 - global_seqlen/min:25542 - global_seqlen/max:34694 - global_seqlen/minmax_diff:9152 - global_seqlen/balanced_min:30320 - global_seqlen/balanced_max:30380 - global_seqlen/mean:30366.25 - actor/entropy:0.19629985094070435 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31755661964416504 - training/rollout_probs_diff_mean:0.0030814865604043007 - training/rollout_probs_diff_std:0.010210192762315273 - training/rollout_actor_probs_pearson_corr:0.9988877177238464 - rollout_corr/rollout_is_mean:0.9999521374702454 - rollout_corr/rollout_is_ratio_fraction_high:1.7170919818454422e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.7170919818454422e-05 - rollout_corr/rollout_is_max:2.3355937004089355 - rollout_corr/rollout_is_min:0.20813822746276855 - rollout_corr/rollout_is_std:0.03025965206325054 - rollout_corr/rollout_is_eff_sample_size:0.9990850720973192 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0012307165889069438 - rollout_corr/rollout_is_seq_max:1.006462574005127 - rollout_corr/rollout_is_seq_min:0.9960255026817322 - rollout_corr/rollout_is_seq_max_deviation:0.006462574005126953 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1838456057012081) - rollout_corr/training_log_ppl:np.float64(0.161605034765671) - rollout_corr/kl:np.float64(0.00040539363182290344) - rollout_corr/k3_kl:np.float64(0.0005141745206458381) - rollout_corr/rollout_ppl:np.float64(1.1833340604789555) - rollout_corr/rollout_log_ppl:np.float64(0.1611996397114126) - rollout_corr/log_ppl_diff:np.float64(0.00040539505425840616) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010806201898958534) - rollout_corr/log_ppl_diff_max:np.float64(0.005684956908226013) - rollout_corr/log_ppl_diff_min:np.float64(-0.0034256726503372192) - rollout_corr/ppl_ratio:np.float64(1.000406396575272) - rollout_corr/chi2_token:np.float64(0.0012630913406610489) - rollout_corr/chi2_seq:np.float64(1.0963045465759933) - actor/pg_loss:np.float64(-1.263362355530262e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005835516549268505) - actor/ppo_kl:np.float64(-5.1322627289351885e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21340569853782654) - perf/mfu/actor:np.float64(0.0874389401304758) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.24371719360352) - actor/lr:np.float64(1e-06) - training/global_step:42 - training/epoch:1 - critic/score/mean:0.5703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0002196446730522439 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0002196446730522439 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:682.4765625 - response_length/max:2365.0 - response_length/min:134.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:682.4765625 - response_length_non_aborted/max:2365.0 - response_length_non_aborted/min:134.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:355.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014196522533893585 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5289256758987904) - timing_s/agent_loop/generate_sequences/max:np.float64(16.370171174407005) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.762006698663754) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.370171174407005) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:2365 - timing_s/gen:22.206629561260343 - timing_s/reward:0.06503966823220253 - timing_s/old_log_prob:2.5655108485370874 - timing_s/adv:0.14332416281104088 - timing_s/update_actor:10.220189092680812 - timing_s/step:35.30416659079492 - timing_s/stop_profile:0.00013269484043121338 - timing_per_token_ms/update_actor:0.04207051040497597 - timing_per_token_ms/gen:0.12710274827008908 - timing_per_token_ms/adv:0.000589981323060309 - perf/total_num_tokens:242930 - perf/time_per_step:35.30416659079492 - perf/throughput:860.1321864348324 (TaskRunner pid=2049544) Training Progress: 42%|████▏ | 42/100 [31:00<53:11, 55.03s/it] (TaskRunner pid=2049544) step:43 - global_seqlen/min:22214 - global_seqlen/max:34661 - global_seqlen/minmax_diff:12447 - global_seqlen/balanced_min:28283 - global_seqlen/balanced_max:28875 - global_seqlen/mean:28428.25 - actor/entropy:0.17894256114959717 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9968691468238831 - training/rollout_probs_diff_mean:0.0030561003368347883 - training/rollout_probs_diff_std:0.010964183136820793 - training/rollout_actor_probs_pearson_corr:0.9986260533332825 - rollout_corr/rollout_is_mean:0.9999217391014099 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.6834687711670995e-05 - rollout_corr/rollout_is_max:1.9516299962997437 - rollout_corr/rollout_is_min:3.1283660064218566e-05 - rollout_corr/rollout_is_std:0.02998857945203781 - rollout_corr/rollout_is_eff_sample_size:0.9991012551699266 - rollout_corr/rollout_is_seq_mean:0.9999701976776123 - rollout_corr/rollout_is_seq_std:0.0015368059976026416 - rollout_corr/rollout_is_seq_max:1.007220983505249 - rollout_corr/rollout_is_seq_min:0.9942285418510437 - rollout_corr/rollout_is_seq_max_deviation:0.0072209835052490234 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1693742866627872) - rollout_corr/training_log_ppl:np.float64(0.15083149004203733) - rollout_corr/kl:np.float64(0.0007233850172774225) - rollout_corr/k3_kl:np.float64(0.0006608226789381888) - rollout_corr/rollout_ppl:np.float64(1.168502108193934) - rollout_corr/rollout_log_ppl:np.float64(0.15010810326202773) - rollout_corr/log_ppl_diff:np.float64(0.000723386780009605) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014408983988687396) - rollout_corr/log_ppl_diff_max:np.float64(0.025284215807914734) - rollout_corr/log_ppl_diff_min:np.float64(-0.005987722426652908) - rollout_corr/ppl_ratio:np.float64(1.0007265941239893) - rollout_corr/chi2_token:np.float64(0.0009258103091269732) - rollout_corr/chi2_seq:np.float64(1.824508806457743) - actor/pg_loss:np.float64(0.0001151182223111391) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000508402825516896) - actor/ppo_kl:np.float64(5.8405713696974004e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2429964765906334) - perf/mfu/actor:np.float64(0.08157217656976501) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.25969696044922) - actor/lr:np.float64(1e-06) - training/global_step:43 - training/epoch:1 - critic/score/mean:0.69140625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69140625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.018738869577646255 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.018738869577646255 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:618.5703125 - response_length/max:2843.0 - response_length/min:133.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:618.5703125 - response_length_non_aborted/max:2843.0 - response_length_non_aborted/min:133.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.8125 - prompt_length/max:365.0 - prompt_length/min:186.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.98380172252655e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.5165256280452013) - timing_s/agent_loop/generate_sequences/max:np.float64(18.731039391830564) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.20822462200158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.731039391830564) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:2843 - timing_s/gen:24.405709667131305 - timing_s/reward:0.0670168325304985 - timing_s/old_log_prob:2.534782351925969 - timing_s/adv:0.14802170917391777 - timing_s/update_actor:10.064561607316136 - timing_s/step:37.307200910523534 - timing_s/stop_profile:0.00010807998478412628 - timing_per_token_ms/update_actor:0.044254226022161655 - timing_per_token_ms/gen:0.15412120734008175 - timing_per_token_ms/adv:0.0006508565826858748 - perf/total_num_tokens:227426 - perf/time_per_step:37.307200910523534 - perf/throughput:762.0043666149454 (TaskRunner pid=2049544) Training Progress: 43%|████▎ | 43/100 [31:37<47:14, 49.73s/it] (TaskRunner pid=2049544) step:44 - global_seqlen/min:22462 - global_seqlen/max:40198 - global_seqlen/minmax_diff:17736 - global_seqlen/balanced_min:31295 - global_seqlen/balanced_max:31374 - global_seqlen/mean:31349.5 - actor/entropy:0.17300812900066376 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6893051266670227 - training/rollout_probs_diff_mean:0.002969349967315793 - training/rollout_probs_diff_std:0.010275507345795631 - training/rollout_actor_probs_pearson_corr:0.9987791776657104 - rollout_corr/rollout_is_mean:0.9999350309371948 - rollout_corr/rollout_is_ratio_fraction_high:1.112074914999539e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.448299659998156e-05 - rollout_corr/rollout_is_max:2.314483165740967 - rollout_corr/rollout_is_min:0.03194784000515938 - rollout_corr/rollout_is_std:0.030064012855291367 - rollout_corr/rollout_is_eff_sample_size:0.9990967333762735 - rollout_corr/rollout_is_seq_mean:0.9999725222587585 - rollout_corr/rollout_is_seq_std:0.0014090986223891377 - rollout_corr/rollout_is_seq_max:1.0068600177764893 - rollout_corr/rollout_is_seq_min:0.9924980998039246 - rollout_corr/rollout_is_seq_max_deviation:0.0075019001960754395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1756953983567655) - rollout_corr/training_log_ppl:np.float64(0.15532817154598888) - rollout_corr/kl:np.float64(0.00048252152897987344) - rollout_corr/k3_kl:np.float64(0.0005270084845747647) - rollout_corr/rollout_ppl:np.float64(1.1751229297369719) - rollout_corr/rollout_log_ppl:np.float64(0.15484564797225175) - rollout_corr/log_ppl_diff:np.float64(0.00048252357373712584) - rollout_corr/log_ppl_abs_diff:np.float64(0.00115150140482001) - rollout_corr/log_ppl_diff_max:np.float64(0.01116035133600235) - rollout_corr/log_ppl_diff_min:np.float64(-0.005915746092796326) - rollout_corr/ppl_ratio:np.float64(1.0004838448949158) - rollout_corr/chi2_token:np.float64(0.0010843609925359488) - rollout_corr/chi2_seq:np.float64(0.9741545785218477) - actor/pg_loss:np.float64(-0.00012030208017677069) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002879910933870633) - actor/ppo_kl:np.float64(-3.409391269992845e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15622974559664726) - perf/mfu/actor:np.float64(0.08872996761202245) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.24708938598633) - actor/lr:np.float64(1e-06) - training/global_step:44 - training/epoch:1 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017787639051675797 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017787639051675797 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:702.515625 - response_length/max:2613.0 - response_length/min:172.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:702.515625 - response_length_non_aborted/max:2613.0 - response_length_non_aborted/min:172.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.15625 - prompt_length/max:460.0 - prompt_length/min:184.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001371428370475769 - timing_s/agent_loop/generate_sequences/min:np.float64(2.046190181747079) - timing_s/agent_loop/generate_sequences/max:np.float64(18.486025908961892) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.959546060519642) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.486025908961892) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:2613 - timing_s/gen:24.597592102363706 - timing_s/reward:0.07184095494449139 - timing_s/old_log_prob:2.5473712403327227 - timing_s/adv:0.13307522423565388 - timing_s/update_actor:10.182054167613387 - timing_s/step:37.62097364850342 - timing_s/stop_profile:0.0001139417290687561 - timing_per_token_ms/update_actor:0.040598949614879774 - timing_per_token_ms/gen:0.13677182503927685 - timing_per_token_ms/adv:0.0005306114301490211 - perf/total_num_tokens:250796 - perf/time_per_step:37.62097364850342 - perf/throughput:833.2984758156863 (TaskRunner pid=2049544) Training Progress: 44%|████▍ | 44/100 [32:15<43:02, 46.11s/it] (TaskRunner pid=2049544) step:45 - global_seqlen/min:20785 - global_seqlen/max:36095 - global_seqlen/minmax_diff:15310 - global_seqlen/balanced_min:29569 - global_seqlen/balanced_max:29783 - global_seqlen/mean:29630.5 - actor/entropy:0.2007351815700531 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4102352559566498 - training/rollout_probs_diff_mean:0.0033135332632809877 - training/rollout_probs_diff_std:0.01088031567633152 - training/rollout_actor_probs_pearson_corr:0.998772919178009 - rollout_corr/rollout_is_mean:1.0001227855682373 - rollout_corr/rollout_is_ratio_fraction_high:6.0338379626045935e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.033837780705653e-05 - rollout_corr/rollout_is_max:5.347679138183594 - rollout_corr/rollout_is_min:0.0047262562438845634 - rollout_corr/rollout_is_std:0.03222286328673363 - rollout_corr/rollout_is_eff_sample_size:0.998963001987778 - rollout_corr/rollout_is_seq_mean:1.00006902217865 - rollout_corr/rollout_is_seq_std:0.001625648234039545 - rollout_corr/rollout_is_seq_max:1.0073453187942505 - rollout_corr/rollout_is_seq_min:0.9958909749984741 - rollout_corr/rollout_is_seq_max_deviation:0.007345318794250488 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.208143400028348) - rollout_corr/training_log_ppl:np.float64(0.17995586975303013) - rollout_corr/kl:np.float64(0.0005010489769254889) - rollout_corr/k3_kl:np.float64(0.0006270660993550337) - rollout_corr/rollout_ppl:np.float64(1.20752296410501) - rollout_corr/rollout_log_ppl:np.float64(0.1794548208345077) - rollout_corr/log_ppl_diff:np.float64(0.0005010489185224287) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013183121991460212) - rollout_corr/log_ppl_diff_max:np.float64(0.006140023469924927) - rollout_corr/log_ppl_diff_min:np.float64(-0.006050072610378265) - rollout_corr/ppl_ratio:np.float64(1.000502604758367) - rollout_corr/chi2_token:np.float64(0.0014851994346827269) - rollout_corr/chi2_seq:np.float64(1.3357745304238051) - actor/pg_loss:np.float64(-6.521574687212706e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00033872173025883967) - actor/ppo_kl:np.float64(-1.556063463681312e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20266040414571762) - perf/mfu/actor:np.float64(0.08389738845778953) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0823974609375) - actor/lr:np.float64(1e-06) - training/global_step:45 - training/epoch:2 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.005806060507893562 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.005806060507893562 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:647.390625 - response_length/max:2551.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:647.390625 - response_length_non_aborted/max:2551.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.5625 - prompt_length/max:437.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00031680427491664886 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9299852326512337) - timing_s/agent_loop/generate_sequences/max:np.float64(16.61781131848693) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.397257668249949) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.61781131848693) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:324 - timing_s/agent_loop/slowest/response_length:2551 - timing_s/gen:22.67037677206099 - timing_s/reward:0.06372027657926083 - timing_s/old_log_prob:2.7545996010303497 - timing_s/adv:0.11587526835501194 - timing_s/update_actor:10.353036934509873 - timing_s/step:36.06465018354356 - timing_s/stop_profile:0.00011578015983104706 - timing_per_token_ms/update_actor:0.04367559159696037 - timing_per_token_ms/gen:0.13678937544988892 - timing_per_token_ms/adv:0.0004888344288613588 - perf/total_num_tokens:237044 - perf/time_per_step:36.06465018354356 - perf/throughput:821.5939943740397 (TaskRunner pid=2049544) Training Progress: 45%|████▌ | 45/100 [32:51<39:37, 43.23s/it] (TaskRunner pid=2049544) step:46 - global_seqlen/min:16300 - global_seqlen/max:40451 - global_seqlen/minmax_diff:24151 - global_seqlen/balanced_min:26008 - global_seqlen/balanced_max:26038 - global_seqlen/mean:26025.75 - actor/entropy:0.16465333104133606 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42921197414398193 - training/rollout_probs_diff_mean:0.0029608916956931353 - training/rollout_probs_diff_std:0.01070943009108305 - training/rollout_actor_probs_pearson_corr:0.9986132979393005 - rollout_corr/rollout_is_mean:0.9999878406524658 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:6.257039058255032e-05 - rollout_corr/rollout_is_max:1.9571115970611572 - rollout_corr/rollout_is_min:0.060181617736816406 - rollout_corr/rollout_is_std:0.030463820323348045 - rollout_corr/rollout_is_eff_sample_size:0.9990728161485534 - rollout_corr/rollout_is_seq_mean:1.000013828277588 - rollout_corr/rollout_is_seq_std:0.0019006438087671995 - rollout_corr/rollout_is_seq_max:1.008240818977356 - rollout_corr/rollout_is_seq_min:0.993937611579895 - rollout_corr/rollout_is_seq_max_deviation:0.008240818977355957 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.153148335404694) - rollout_corr/training_log_ppl:np.float64(0.136645665843389) - rollout_corr/kl:np.float64(0.0006094530078737392) - rollout_corr/k3_kl:np.float64(0.0006117465548527434) - rollout_corr/rollout_ppl:np.float64(1.152437952812761) - rollout_corr/rollout_log_ppl:np.float64(0.13603621201764327) - rollout_corr/log_ppl_diff:np.float64(0.0006094538257457316) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014239892771001905) - rollout_corr/log_ppl_diff_max:np.float64(0.014275498688220978) - rollout_corr/log_ppl_diff_min:np.float64(-0.006409347057342529) - rollout_corr/ppl_ratio:np.float64(1.0006118323653936) - rollout_corr/chi2_token:np.float64(0.0012153126299381256) - rollout_corr/chi2_seq:np.float64(0.35479475394822657) - actor/pg_loss:np.float64(-5.846214480698109e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001732110890770855) - actor/ppo_kl:np.float64(5.357556297269639e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1466388632543385) - perf/mfu/actor:np.float64(0.07518990372656345) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.19413375854492) - actor/lr:np.float64(1e-06) - training/global_step:46 - training/epoch:2 - critic/score/mean:0.7578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.023664643988013268 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.023664643988013268 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:561.8671875 - response_length/max:2403.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:561.8671875 - response_length_non_aborted/max:2403.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.4375 - prompt_length/max:337.0 - prompt_length/min:158.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0002136826515197754 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7386901136487722) - timing_s/agent_loop/generate_sequences/max:np.float64(15.293935690075159) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.345510819919582) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.293935690075159) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:2403 - timing_s/gen:21.075742207467556 - timing_s/reward:0.06618223898112774 - timing_s/old_log_prob:2.503137046471238 - timing_s/adv:0.11277000606060028 - timing_s/update_actor:10.213601699098945 - timing_s/step:34.05827110260725 - timing_s/stop_profile:0.00012739188969135284 - timing_per_token_ms/update_actor:0.04905527073714948 - timing_per_token_ms/gen:0.14652416056582793 - timing_per_token_ms/adv:0.000541627071557017 - perf/total_num_tokens:208206 - perf/time_per_step:34.05827110260725 - perf/throughput:764.1535861169318 (TaskRunner pid=2049544) Training Progress: 46%|████▌ | 46/100 [33:25<36:26, 40.50s/it] (TaskRunner pid=2049544) step:47 - global_seqlen/min:23716 - global_seqlen/max:42058 - global_seqlen/minmax_diff:18342 - global_seqlen/balanced_min:31374 - global_seqlen/balanced_max:31661 - global_seqlen/mean:31456.375 - actor/entropy:0.14881354570388794 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4682484567165375 - training/rollout_probs_diff_mean:0.002767270663753152 - training/rollout_probs_diff_std:0.010147105902433395 - training/rollout_actor_probs_pearson_corr:0.9986124634742737 - rollout_corr/rollout_is_mean:1.0000232458114624 - rollout_corr/rollout_is_ratio_fraction_high:5.438477273855824e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.350781819084659e-05 - rollout_corr/rollout_is_max:2.263822555541992 - rollout_corr/rollout_is_min:0.43962690234184265 - rollout_corr/rollout_is_std:0.02890361100435257 - rollout_corr/rollout_is_eff_sample_size:0.999165278640662 - rollout_corr/rollout_is_seq_mean:1.0001194477081299 - rollout_corr/rollout_is_seq_std:0.0015050210058689117 - rollout_corr/rollout_is_seq_max:1.005627155303955 - rollout_corr/rollout_is_seq_min:0.993998646736145 - rollout_corr/rollout_is_seq_max_deviation:0.0060013532638549805 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1467792866751552) - rollout_corr/training_log_ppl:np.float64(0.13201592379482463) - rollout_corr/kl:np.float64(0.0004759397659170883) - rollout_corr/k3_kl:np.float64(0.0005555009257420807) - rollout_corr/rollout_ppl:np.float64(1.146229465957731) - rollout_corr/rollout_log_ppl:np.float64(0.13153998326015426) - rollout_corr/log_ppl_diff:np.float64(0.00047594053467037156) - rollout_corr/log_ppl_abs_diff:np.float64(0.001261575052922126) - rollout_corr/log_ppl_diff_max:np.float64(0.008257180452346802) - rollout_corr/log_ppl_diff_min:np.float64(-0.0055009350180625916) - rollout_corr/ppl_ratio:np.float64(1.000477674184367) - rollout_corr/chi2_token:np.float64(0.0013230419717729092) - rollout_corr/chi2_seq:np.float64(0.636790500022471) - actor/pg_loss:np.float64(-0.00010656809899955988) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00031795622612662555) - actor/ppo_kl:np.float64(0.00011273976376457995) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1950441598892212) - perf/mfu/actor:np.float64(0.0887384677499009) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.28681945800781) - actor/lr:np.float64(1e-06) - training/global_step:47 - training/epoch:2 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.011680489405989647 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.011680489405989647 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:718.26171875 - response_length/max:4213.0 - response_length/min:71.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:718.26171875 - response_length_non_aborted/max:4213.0 - response_length_non_aborted/min:71.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.75 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00017112679779529572 - timing_s/agent_loop/generate_sequences/min:np.float64(0.828915087506175) - timing_s/agent_loop/generate_sequences/max:np.float64(25.74352060444653) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.723139668560179) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.74352060444653) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:311 - timing_s/agent_loop/slowest/response_length:4213 - timing_s/gen:31.583294926211238 - timing_s/reward:0.0717991292476654 - timing_s/old_log_prob:2.5992581117898226 - timing_s/adv:0.12912281975150108 - timing_s/update_actor:10.13827026821673 - timing_s/step:44.60779097676277 - timing_s/stop_profile:0.00012019835412502289 - timing_per_token_ms/update_actor:0.040287025556094466 - timing_per_token_ms/gen:0.1717650301901359 - timing_per_token_ms/adv:0.0005131027484552061 - perf/total_num_tokens:251651 - perf/time_per_step:44.60779097676277 - perf/throughput:705.1767036925535 (TaskRunner pid=2049544) Training Progress: 47%|████▋ | 47/100 [34:10<36:52, 41.75s/it] (TaskRunner pid=2049544) step:48 - global_seqlen/min:18590 - global_seqlen/max:34823 - global_seqlen/minmax_diff:16233 - global_seqlen/balanced_min:28237 - global_seqlen/balanced_max:28801 - global_seqlen/mean:28407.875 - actor/entropy:0.14581364393234253 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.47068536281585693 - training/rollout_probs_diff_mean:0.00291546992957592 - training/rollout_probs_diff_std:0.01052320096641779 - training/rollout_actor_probs_pearson_corr:0.9985108971595764 - rollout_corr/rollout_is_mean:0.9999670386314392 - rollout_corr/rollout_is_ratio_fraction_high:1.2597076420206577e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.779122926061973e-05 - rollout_corr/rollout_is_max:2.1663451194763184 - rollout_corr/rollout_is_min:0.35977837443351746 - rollout_corr/rollout_is_std:0.030446205288171768 - rollout_corr/rollout_is_eff_sample_size:0.9990738870447088 - rollout_corr/rollout_is_seq_mean:0.9999531507492065 - rollout_corr/rollout_is_seq_std:0.0016499547054991126 - rollout_corr/rollout_is_seq_max:1.0073652267456055 - rollout_corr/rollout_is_seq_min:0.9916918873786926 - rollout_corr/rollout_is_seq_max_deviation:0.008308112621307373 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1378675876185298) - rollout_corr/training_log_ppl:np.float64(0.12675013709667837) - rollout_corr/kl:np.float64(0.000753273961016987) - rollout_corr/k3_kl:np.float64(0.0006820533353391056) - rollout_corr/rollout_ppl:np.float64(1.1369990683160722) - rollout_corr/rollout_log_ppl:np.float64(0.12599686199246207) - rollout_corr/log_ppl_diff:np.float64(0.0007532751042163) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014732455310877413) - rollout_corr/log_ppl_diff_max:np.float64(0.009199053049087524) - rollout_corr/log_ppl_diff_min:np.float64(-0.006944321095943451) - rollout_corr/ppl_ratio:np.float64(1.0007553710602224) - rollout_corr/chi2_token:np.float64(0.0013299805577844381) - rollout_corr/chi2_seq:np.float64(0.6524860400240868) - actor/pg_loss:np.float64(-2.4408800527453423e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0007820084601917188) - actor/ppo_kl:np.float64(0.00018927762639009416) - actor/pg_clipfrac_lower:np.float64(1.1097301467088982e-05) - actor/grad_norm:np.float64(0.2395218200981617) - perf/mfu/actor:np.float64(0.08095804530180152) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.21407699584961) - actor/lr:np.float64(1e-06) - training/global_step:48 - training/epoch:2 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01809648796916008 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01809648796916008 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:620.18359375 - response_length/max:3666.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:620.18359375 - response_length_non_aborted/max:3666.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.5625 - prompt_length/max:365.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014730170369148254 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9394192546606064) - timing_s/agent_loop/generate_sequences/max:np.float64(21.90096348337829) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.9833613064474775) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.90096348337829) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:3666 - timing_s/gen:28.045954892411828 - timing_s/reward:0.0676717646420002 - timing_s/old_log_prob:2.6066879238933325 - timing_s/adv:0.12023183144629002 - timing_s/update_actor:10.316766967996955 - timing_s/step:41.24971695430577 - timing_s/stop_profile:0.00013453327119350433 - timing_per_token_ms/update_actor:0.04539571759589971 - timing_per_token_ms/gen:0.17664851570170015 - timing_per_token_ms/adv:0.0005290427013912956 - perf/total_num_tokens:227263 - perf/time_per_step:41.24971695430577 - perf/throughput:688.680483104132 (TaskRunner pid=2049544) Training Progress: 48%|████▊ | 48/100 [34:51<36:03, 41.61s/it] (TaskRunner pid=2049544) step:49 - global_seqlen/min:18126 - global_seqlen/max:38899 - global_seqlen/minmax_diff:20773 - global_seqlen/balanced_min:26338 - global_seqlen/balanced_max:26710 - global_seqlen/mean:26451.75 - actor/entropy:0.18256059288978577 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3656392991542816 - training/rollout_probs_diff_mean:0.0032566494774073362 - training/rollout_probs_diff_std:0.010963589884340763 - training/rollout_actor_probs_pearson_corr:0.9987143278121948 - rollout_corr/rollout_is_mean:1.0001035928726196 - rollout_corr/rollout_is_ratio_fraction_high:6.82920153849409e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.146281521068886e-05 - rollout_corr/rollout_is_max:2.4023168087005615 - rollout_corr/rollout_is_min:0.14749553799629211 - rollout_corr/rollout_is_std:0.03197216987609863 - rollout_corr/rollout_is_eff_sample_size:0.9989789431740315 - rollout_corr/rollout_is_seq_mean:1.0002081394195557 - rollout_corr/rollout_is_seq_std:0.001892627333290875 - rollout_corr/rollout_is_seq_max:1.008901834487915 - rollout_corr/rollout_is_seq_min:0.9917373061180115 - rollout_corr/rollout_is_seq_max_deviation:0.008901834487915039 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1714685205370188) - rollout_corr/training_log_ppl:np.float64(0.15115834452444687) - rollout_corr/kl:np.float64(0.0004743365148804912) - rollout_corr/k3_kl:np.float64(0.0006857500739982925) - rollout_corr/rollout_ppl:np.float64(1.1708842911757529) - rollout_corr/rollout_log_ppl:np.float64(0.15068400763266254) - rollout_corr/log_ppl_diff:np.float64(0.0004743368917843327) - rollout_corr/log_ppl_abs_diff:np.float64(0.001488212525146082) - rollout_corr/log_ppl_diff_max:np.float64(0.010617084801197052) - rollout_corr/log_ppl_diff_min:np.float64(-0.015819266438484192) - rollout_corr/ppl_ratio:np.float64(1.0004769298247993) - rollout_corr/chi2_token:np.float64(0.0018537091091275215) - rollout_corr/chi2_seq:np.float64(0.6309683343861252) - actor/pg_loss:np.float64(3.3562071621418e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005490989437930693) - actor/ppo_kl:np.float64(8.648514501530258e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2604895681142807) - perf/mfu/actor:np.float64(0.07564944214215971) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.27551984786987) - actor/lr:np.float64(1e-06) - training/global_step:49 - training/epoch:2 - critic/score/mean:0.77734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.77734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015421191230416298 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015421191230416298 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:571.9921875 - response_length/max:3613.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.9921875 - response_length_non_aborted/max:3613.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:254.625 - prompt_length/max:362.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014311634004116058 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9619488045573235) - timing_s/agent_loop/generate_sequences/max:np.float64(21.043767675757408) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.410715061574592) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.043767675757408) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:3586 - timing_s/gen:27.190765019506216 - timing_s/reward:0.06571215577423573 - timing_s/old_log_prob:2.4588573779910803 - timing_s/adv:0.11915759928524494 - timing_s/update_actor:10.26756145618856 - timing_s/step:40.190888445824385 - timing_s/stop_profile:0.00011413171887397766 - timing_per_token_ms/update_actor:0.048520237111857246 - timing_per_token_ms/gen:0.185691217779869 - timing_per_token_ms/adv:0.0005630893952443834 - perf/total_num_tokens:211614 - perf/time_per_step:40.190888445824385 - perf/throughput:658.1529053694804 (TaskRunner pid=2049544) Training Progress: 49%|████▉ | 49/100 [35:31<35:01, 41.20s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 50} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) - $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs ($ 1 \, \mathrm{pC} = 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_40/actor (TaskRunner pid=2049544) step:50 - global_seqlen/min:22621 - global_seqlen/max:38182 - global_seqlen/minmax_diff:15561 - global_seqlen/balanced_min:31307 - global_seqlen/balanced_max:32242 - global_seqlen/mean:31432.0 - actor/entropy:0.20185792446136475 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3018432855606079 - training/rollout_probs_diff_mean:0.003106999211013317 - training/rollout_probs_diff_std:0.010115313343703747 - training/rollout_actor_probs_pearson_corr:0.9989494681358337 - rollout_corr/rollout_is_mean:1.0000050067901611 - rollout_corr/rollout_is_ratio_fraction_high:5.458515261125285e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.458515261125285e-06 - rollout_corr/rollout_is_max:2.0969016551971436 - rollout_corr/rollout_is_min:0.33815088868141174 - rollout_corr/rollout_is_std:0.030253741890192032 - rollout_corr/rollout_is_eff_sample_size:0.9990853100798829 - rollout_corr/rollout_is_seq_mean:0.9999303817749023 - rollout_corr/rollout_is_seq_std:0.001567038125358522 - rollout_corr/rollout_is_seq_max:1.0050349235534668 - rollout_corr/rollout_is_seq_min:0.993672788143158 - rollout_corr/rollout_is_seq_max_deviation:0.006327211856842041 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.191426008939743) - rollout_corr/training_log_ppl:np.float64(0.16859979891887633) - rollout_corr/kl:np.float64(0.0005692955341736194) - rollout_corr/k3_kl:np.float64(0.0005476311655883137) - rollout_corr/rollout_ppl:np.float64(1.1907334956340492) - rollout_corr/rollout_log_ppl:np.float64(0.16803050318412716) - rollout_corr/log_ppl_diff:np.float64(0.0005692957347491756) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012659797939704731) - rollout_corr/log_ppl_diff_max:np.float64(0.008102566003799438) - rollout_corr/log_ppl_diff_min:np.float64(-0.00437454879283905) - rollout_corr/ppl_ratio:np.float64(1.0005708439275622) - rollout_corr/chi2_token:np.float64(0.0010575931519269943) - rollout_corr/chi2_seq:np.float64(1.1911898551043123) - actor/pg_loss:np.float64(6.677815690636635e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00036017004458699375) - actor/ppo_kl:np.float64(2.6112116451315615e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17873636819422245) - perf/mfu/actor:np.float64(0.0885810936633662) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.26837921142578) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/reward/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09966534252010995) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8278625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.06058065805524353) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06833156224087977) - val-aux/sciknoweval/score/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/score/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/score/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/score/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09966534252010995) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8278625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.06058065805524353) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6779125) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06833156224087977) - val-core/sciknoweval/acc/mean@16:np.float64(0.6453125) - val-aux/sciknoweval/acc/std@16:np.float64(0.17485248493267602) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7153499999999999) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.13834881246475225) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.57635) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1582729828765162) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6450625000000001) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17474490419708083) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7678750000000001) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.09578158475998025) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5130625) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12920767926064702) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6637124999999999) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13847719028530886) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8036125000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06368274289112907) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.458575) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.09684683458113633) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.674125) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09966534252010995) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8278625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.03910595287238987) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.4183625) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.06058065805524353) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6779125) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06833156224087977) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.978125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.05947208208072883) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9967624999999998) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.02126663775002429) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9593875000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.0761225622474269) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9781625) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.059091934676165636) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0017437187447881502) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9295375) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08943279499033274) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9918500000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03609564352248361) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.890025) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.09147875280510498) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9984124999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.014058383946768527) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8474) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.07545396833360063) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.99985) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.002693900935493915) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:50 - training/epoch:2 - critic/score/mean:0.6171875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6171875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016236353665590286 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016236353665590286 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:715.625 - response_length/max:3780.0 - response_length/min:93.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:715.625 - response_length_non_aborted/max:3780.0 - response_length_non_aborted/min:93.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.625 - prompt_length/max:355.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014733336865901947 - timing_s/agent_loop/generate_sequences/min:np.float64(0.98030811175704) - timing_s/agent_loop/generate_sequences/max:np.float64(23.301443008705974) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.870220143442566) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.301443008705974) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:237 - timing_s/agent_loop/slowest/response_length:3780 - timing_s/gen:29.260056165978312 - timing_s/reward:0.07223350368440151 - timing_s/old_log_prob:2.538018411025405 - timing_s/adv:0.11729281768202782 - timing_s/update_actor:10.420197248458862 - timing_s/step:42.49460377730429 - timing_s/testing:117.82088489830494 - timing_s/save_checkpoint:6.728960711508989 - timing_s/stop_profile:0.00011482089757919312 - timing_per_token_ms/update_actor:0.04143944566229822 - timing_per_token_ms/gen:0.15971646378809123 - timing_per_token_ms/adv:0.0004664546389110931 - perf/total_num_tokens:251456 - perf/time_per_step:42.49460377730429 - perf/throughput:739.6703864971049 (TaskRunner pid=2049544) Training Progress: 50%|█████ | 50/100 [38:18<1:05:48, 78.97s/it] (TaskRunner pid=2049544) step:51 - global_seqlen/min:18785 - global_seqlen/max:33808 - global_seqlen/minmax_diff:15023 - global_seqlen/balanced_min:26399 - global_seqlen/balanced_max:26471 - global_seqlen/mean:26450.125 - actor/entropy:0.24430422484874725 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4564788341522217 - training/rollout_probs_diff_mean:0.003582285949960351 - training/rollout_probs_diff_std:0.010883934795856476 - training/rollout_actor_probs_pearson_corr:0.999004602432251 - rollout_corr/rollout_is_mean:0.999866783618927 - rollout_corr/rollout_is_ratio_fraction_high:1.396638253936544e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.189914761809632e-05 - rollout_corr/rollout_is_max:2.2966716289520264 - rollout_corr/rollout_is_min:0.327895849943161 - rollout_corr/rollout_is_std:0.033104244619607925 - rollout_corr/rollout_is_eff_sample_size:0.9989051897151183 - rollout_corr/rollout_is_seq_mean:0.9998487830162048 - rollout_corr/rollout_is_seq_std:0.0018385471776127815 - rollout_corr/rollout_is_seq_max:1.0075657367706299 - rollout_corr/rollout_is_seq_min:0.9936858415603638 - rollout_corr/rollout_is_seq_max_deviation:0.007565736770629883 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.222205103840679) - rollout_corr/training_log_ppl:np.float64(0.18761249752424192) - rollout_corr/kl:np.float64(0.0007189881607398974) - rollout_corr/k3_kl:np.float64(0.0005682342473676272) - rollout_corr/rollout_ppl:np.float64(1.2213212260976434) - rollout_corr/rollout_log_ppl:np.float64(0.1868935083039105) - rollout_corr/log_ppl_diff:np.float64(0.000718989220331423) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014408860151888803) - rollout_corr/log_ppl_diff_max:np.float64(0.007139421999454498) - rollout_corr/log_ppl_diff_min:np.float64(-0.005957402288913727) - rollout_corr/ppl_ratio:np.float64(1.0007210047915578) - rollout_corr/chi2_token:np.float64(0.0008357972837984562) - rollout_corr/chi2_seq:np.float64(1.0313024234492332) - actor/pg_loss:np.float64(4.522060044109821e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003210717095498694) - actor/ppo_kl:np.float64(-9.128155866733323e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14451957494020462) - perf/mfu/actor:np.float64(0.07523053933355461) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.30613327026367) - actor/lr:np.float64(1e-06) - training/global_step:51 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011186200194060802 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011186200194060802 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:559.37890625 - response_length/max:2702.0 - response_length/min:100.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:559.37890625 - response_length_non_aborted/max:2702.0 - response_length_non_aborted/min:100.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.1875 - prompt_length/max:356.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.59448516368866e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9070874620229006) - timing_s/agent_loop/generate_sequences/max:np.float64(16.53190640732646) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.214220573740022) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.53190640732646) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:2702 - timing_s/gen:22.698202308267355 - timing_s/reward:0.06384330801665783 - timing_s/old_log_prob:2.640516960993409 - timing_s/adv:0.15581872314214706 - timing_s/update_actor:10.303586572408676 - timing_s/step:35.9570118393749 - timing_s/stop_profile:0.00012137554585933685 - timing_per_token_ms/update_actor:0.048693468236958595 - timing_per_token_ms/gen:0.1585058924746849 - timing_per_token_ms/adv:0.0007363798996325493 - perf/total_num_tokens:211601 - perf/time_per_step:35.9570118393749 - perf/throughput:735.6040907447059 (TaskRunner pid=2049544) Training Progress: 51%|█████ | 51/100 [38:54<53:58, 66.09s/it] (TaskRunner pid=2049544) step:52 - global_seqlen/min:23429 - global_seqlen/max:36160 - global_seqlen/minmax_diff:12731 - global_seqlen/balanced_min:30668 - global_seqlen/balanced_max:30765 - global_seqlen/mean:30739.125 - actor/entropy:0.22692662477493286 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3938141167163849 - training/rollout_probs_diff_mean:0.0033940812572836876 - training/rollout_probs_diff_std:0.010467931628227234 - training/rollout_actor_probs_pearson_corr:0.9990243315696716 - rollout_corr/rollout_is_mean:1.000043272972107 - rollout_corr/rollout_is_ratio_fraction_high:1.1359441487002186e-05 - rollout_corr/rollout_is_ratio_fraction_low:2.2718882974004373e-05 - rollout_corr/rollout_is_max:2.0554285049438477 - rollout_corr/rollout_is_min:0.29720351099967957 - rollout_corr/rollout_is_std:0.03246244043111801 - rollout_corr/rollout_is_eff_sample_size:0.9989472992270986 - rollout_corr/rollout_is_seq_mean:1.0000196695327759 - rollout_corr/rollout_is_seq_std:0.001915202010422945 - rollout_corr/rollout_is_seq_max:1.0059826374053955 - rollout_corr/rollout_is_seq_min:0.9903362989425659 - rollout_corr/rollout_is_seq_max_deviation:0.009663701057434082 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.231195269152522) - rollout_corr/training_log_ppl:np.float64(0.19834982651809696) - rollout_corr/kl:np.float64(0.0008127379676388102) - rollout_corr/k3_kl:np.float64(0.0006191496557619303) - rollout_corr/rollout_ppl:np.float64(1.230196955613792) - rollout_corr/rollout_log_ppl:np.float64(0.19753708717325935) - rollout_corr/log_ppl_diff:np.float64(0.0008127393448376097) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014782494035898708) - rollout_corr/log_ppl_diff_max:np.float64(0.011166423559188843) - rollout_corr/log_ppl_diff_min:np.float64(-0.00570167601108551) - rollout_corr/ppl_ratio:np.float64(1.0008151046931744) - rollout_corr/chi2_token:np.float64(0.0008716660086065531) - rollout_corr/chi2_seq:np.float64(0.30706389155238867) - actor/pg_loss:np.float64(2.5613000616431236e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015267897435933264) - actor/ppo_kl:np.float64(0.00024156836637168055) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18445459008216858) - perf/mfu/actor:np.float64(0.08808419007285372) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.37739562988281) - actor/lr:np.float64(1e-06) - training/global_step:52 - training/epoch:2 - critic/score/mean:0.65625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007727969903498888 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007727969903498888 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:687.75390625 - response_length/max:2764.0 - response_length/min:70.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:687.75390625 - response_length_non_aborted/max:2764.0 - response_length_non_aborted/min:70.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:272.84375 - prompt_length/max:365.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011474080383777618 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7448907978832722) - timing_s/agent_loop/generate_sequences/max:np.float64(18.379133939743042) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.580681977618951) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.379133939743042) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:344 - timing_s/agent_loop/slowest/response_length:2764 - timing_s/gen:24.19850930571556 - timing_s/reward:0.07030453719198704 - timing_s/old_log_prob:2.519341915845871 - timing_s/adv:0.11268062144517899 - timing_s/update_actor:10.240152211859822 - timing_s/step:37.230506498366594 - timing_s/stop_profile:0.00011861883103847504 - timing_per_token_ms/update_actor:0.0416413618306467 - timing_per_token_ms/gen:0.13744077077054248 - timing_per_token_ms/adv:0.0004582133577532663 - perf/total_num_tokens:245913 - perf/time_per_step:37.230506498366594 - perf/throughput:825.643481410832 (TaskRunner pid=2049544) Training Progress: 52%|█████▏ | 52/100 [39:32<45:57, 57.45s/it] (TaskRunner pid=2049544) step:53 - global_seqlen/min:19572 - global_seqlen/max:34404 - global_seqlen/minmax_diff:14832 - global_seqlen/balanced_min:27920 - global_seqlen/balanced_max:29013 - global_seqlen/mean:28188.25 - actor/entropy:0.22438420355319977 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30675768852233887 - training/rollout_probs_diff_mean:0.0035303884651511908 - training/rollout_probs_diff_std:0.010939665138721466 - training/rollout_actor_probs_pearson_corr:0.9988002181053162 - rollout_corr/rollout_is_mean:1.000083327293396 - rollout_corr/rollout_is_ratio_fraction_high:1.938861169037409e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.524009636952542e-05 - rollout_corr/rollout_is_max:2.4050400257110596 - rollout_corr/rollout_is_min:0.11861640214920044 - rollout_corr/rollout_is_std:0.03258340060710907 - rollout_corr/rollout_is_eff_sample_size:0.9989399238583818 - rollout_corr/rollout_is_seq_mean:1.0001201629638672 - rollout_corr/rollout_is_seq_std:0.001550388871692121 - rollout_corr/rollout_is_seq_max:1.0046480894088745 - rollout_corr/rollout_is_seq_min:0.994028627872467 - rollout_corr/rollout_is_seq_max_deviation:0.005971372127532959 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2066223253495991) - rollout_corr/training_log_ppl:np.float64(0.17772597989824135) - rollout_corr/kl:np.float64(0.0005446721567352597) - rollout_corr/k3_kl:np.float64(0.0006030060052921726) - rollout_corr/rollout_ppl:np.float64(1.2059485423378646) - rollout_corr/rollout_log_ppl:np.float64(0.17718130597495474) - rollout_corr/log_ppl_diff:np.float64(0.0005446739232866094) - rollout_corr/log_ppl_abs_diff:np.float64(0.001306041274801828) - rollout_corr/log_ppl_diff_max:np.float64(0.006498947739601135) - rollout_corr/log_ppl_diff_min:np.float64(-0.005862616002559662) - rollout_corr/ppl_ratio:np.float64(1.0005462081171572) - rollout_corr/chi2_token:np.float64(0.001310169231146574) - rollout_corr/chi2_seq:np.float64(0.6170898247510195) - actor/pg_loss:np.float64(5.588738713413477e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00040137838948339777) - actor/ppo_kl:np.float64(0.00010251342614253645) - actor/pg_clipfrac_lower:np.float64(1.1289739632047713e-06) - actor/grad_norm:np.float64(0.2714943513274193) - perf/mfu/actor:np.float64(0.0789630435605575) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.31622695922852) - actor/lr:np.float64(1e-06) - training/global_step:53 - training/epoch:2 - critic/score/mean:0.71484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.71484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02330026589334011 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02330026589334011 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:604.4140625 - response_length/max:3739.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:604.4140625 - response_length_non_aborted/max:3739.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.46875 - prompt_length/max:375.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001577865332365036 - timing_s/agent_loop/generate_sequences/min:np.float64(1.263271963223815) - timing_s/agent_loop/generate_sequences/max:np.float64(22.0587509367615) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.729596399491129) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.0587509367615) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3739 - timing_s/gen:27.900284377858043 - timing_s/reward:0.0937537681311369 - timing_s/old_log_prob:2.7064144033938646 - timing_s/adv:0.1434136312454939 - timing_s/update_actor:10.502349453046918 - timing_s/step:41.45142296142876 - timing_s/stop_profile:0.00011624768376350403 - timing_per_token_ms/update_actor:0.04657237258896401 - timing_per_token_ms/gen:0.18031593341858748 - timing_per_token_ms/adv:0.0006359637049368704 - perf/total_num_tokens:225506 - perf/time_per_step:41.45142296142876 - perf/throughput:680.030937085794 (TaskRunner pid=2049544) Training Progress: 53%|█████▎ | 53/100 [40:13<41:15, 52.67s/it] (TaskRunner pid=2049544) step:54 - global_seqlen/min:21471 - global_seqlen/max:31367 - global_seqlen/minmax_diff:9896 - global_seqlen/balanced_min:26297 - global_seqlen/balanced_max:26328 - global_seqlen/mean:26318.0 - actor/entropy:0.21818441152572632 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292486190795898 - training/rollout_probs_diff_mean:0.0032305221538990736 - training/rollout_probs_diff_std:0.010491388849914074 - training/rollout_actor_probs_pearson_corr:0.9989482760429382 - rollout_corr/rollout_is_mean:1.0000078678131104 - rollout_corr/rollout_is_ratio_fraction_high:2.0765270164702088e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.3843512533640023e-05 - rollout_corr/rollout_is_max:2.25616192817688 - rollout_corr/rollout_is_min:0.26139989495277405 - rollout_corr/rollout_is_std:0.03054003044962883 - rollout_corr/rollout_is_eff_sample_size:0.999068175625075 - rollout_corr/rollout_is_seq_mean:0.9999727010726929 - rollout_corr/rollout_is_seq_std:0.001957092434167862 - rollout_corr/rollout_is_seq_max:1.007343053817749 - rollout_corr/rollout_is_seq_min:0.9894382953643799 - rollout_corr/rollout_is_seq_max_deviation:0.010561704635620117 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1926028821617365) - rollout_corr/training_log_ppl:np.float64(0.16857526991225313) - rollout_corr/kl:np.float64(0.0005841684274781045) - rollout_corr/k3_kl:np.float64(0.0005794948536674838) - rollout_corr/rollout_ppl:np.float64(1.1918872729875147) - rollout_corr/rollout_log_ppl:np.float64(0.1679911007740884) - rollout_corr/log_ppl_diff:np.float64(0.0005841691381647252) - rollout_corr/log_ppl_abs_diff:np.float64(0.001538367178000044) - rollout_corr/log_ppl_diff_max:np.float64(0.011501207947731018) - rollout_corr/log_ppl_diff_min:np.float64(-0.005205966532230377) - rollout_corr/ppl_ratio:np.float64(1.0005866582505405) - rollout_corr/chi2_token:np.float64(0.0011341231875121593) - rollout_corr/chi2_seq:np.float64(1.4627508698031306) - actor/pg_loss:np.float64(0.00015633960720151663) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00042431396536812827) - actor/ppo_kl:np.float64(3.235917171728886e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16759618930518627) - perf/mfu/actor:np.float64(0.07638956882275981) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.35180282592773) - actor/lr:np.float64(1e-06) - training/global_step:54 - training/epoch:2 - critic/score/mean:0.73828125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.73828125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008146907202899456 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008146907202899456 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:564.34375 - response_length/max:2093.0 - response_length/min:76.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:564.34375 - response_length_non_aborted/max:2093.0 - response_length_non_aborted/min:76.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.09375 - prompt_length/max:363.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015513785183429718 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6626291759312153) - timing_s/agent_loop/generate_sequences/max:np.float64(14.199840543791652) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.107884573328192) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(14.199840543791652) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:326 - timing_s/agent_loop/slowest/response_length:2093 - timing_s/gen:19.986812436953187 - timing_s/reward:0.06590797752141953 - timing_s/old_log_prob:2.560090212151408 - timing_s/adv:0.13934014923870564 - timing_s/update_actor:10.21920620650053 - timing_s/step:33.05696338787675 - timing_s/stop_profile:0.00011862441897392273 - timing_per_token_ms/update_actor:0.04853715236007927 - timing_per_token_ms/gen:0.13834384819863493 - timing_per_token_ms/adv:0.0006618101168340378 - perf/total_num_tokens:210544 - perf/time_per_step:33.05696338787675 - perf/throughput:796.1408823670663 (TaskRunner pid=2049544) Training Progress: 54%|█████▍ | 54/100 [40:46<35:52, 46.80s/it] (TaskRunner pid=2049544) step:55 - global_seqlen/min:18191 - global_seqlen/max:33987 - global_seqlen/minmax_diff:15796 - global_seqlen/balanced_min:26610 - global_seqlen/balanced_max:26684 - global_seqlen/mean:26660.125 - actor/entropy:0.1736723929643631 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23105892539024353 - training/rollout_probs_diff_mean:0.0030401991680264473 - training/rollout_probs_diff_std:0.01039278693497181 - training/rollout_actor_probs_pearson_corr:0.9987287521362305 - rollout_corr/rollout_is_mean:1.0000579357147217 - rollout_corr/rollout_is_ratio_fraction_high:1.375430656480603e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.375430656480603e-05 - rollout_corr/rollout_is_max:2.335618734359741 - rollout_corr/rollout_is_min:0.392137348651886 - rollout_corr/rollout_is_std:0.030324585735797882 - rollout_corr/rollout_is_eff_sample_size:0.999081502372468 - rollout_corr/rollout_is_seq_mean:1.000092625617981 - rollout_corr/rollout_is_seq_std:0.00183792260941118 - rollout_corr/rollout_is_seq_max:1.0104801654815674 - rollout_corr/rollout_is_seq_min:0.9918577671051025 - rollout_corr/rollout_is_seq_max_deviation:0.010480165481567383 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.1669259667396545) - rollout_corr/training_log_ppl:np.float64(0.14700542857826804) - rollout_corr/kl:np.float64(0.0007857797350844242) - rollout_corr/k3_kl:np.float64(0.0007165672559068526) - rollout_corr/rollout_ppl:np.float64(1.1659750957041979) - rollout_corr/rollout_log_ppl:np.float64(0.14621964823891176) - rollout_corr/log_ppl_diff:np.float64(0.000785780339356279) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016149269104062114) - rollout_corr/log_ppl_diff_max:np.float64(0.058644767850637436) - rollout_corr/log_ppl_diff_min:np.float64(-0.004740942269563675) - rollout_corr/ppl_ratio:np.float64(1.00079559488222) - rollout_corr/chi2_token:np.float64(0.0010453276336193085) - rollout_corr/chi2_seq:np.float64(0.6579223412554711) - actor/pg_loss:np.float64(-0.00018631992861628532) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0005601525168685839) - actor/ppo_kl:np.float64(0.00037079197026113064) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.25370683521032333) - perf/mfu/actor:np.float64(0.0767427855529022) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.34751892089844) - actor/lr:np.float64(1e-06) - training/global_step:55 - training/epoch:2 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0002467178856022656 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0002467178856022656 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:568.00390625 - response_length/max:2618.0 - response_length/min:92.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:568.00390625 - response_length_non_aborted/max:2618.0 - response_length_non_aborted/min:92.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.125 - prompt_length/max:383.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001354403793811798 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8230457752943039) - timing_s/agent_loop/generate_sequences/max:np.float64(16.547815980389714) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.079917931871023) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.547815980389714) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:2618 - timing_s/gen:22.890313317999244 - timing_s/reward:0.06661977060139179 - timing_s/old_log_prob:2.436913600191474 - timing_s/adv:0.1175360195338726 - timing_s/update_actor:10.17019310221076 - timing_s/step:35.7683734241873 - timing_s/stop_profile:0.0001267455518245697 - timing_per_token_ms/update_actor:0.04768447776506468 - timing_per_token_ms/gen:0.15742019626019876 - timing_per_token_ms/adv:0.0005510852796726975 - perf/total_num_tokens:213281 - perf/time_per_step:35.7683734241873 - perf/throughput:745.3546932042451 (TaskRunner pid=2049544) Training Progress: 55%|█████▌ | 55/100 [41:22<32:37, 43.51s/it] (TaskRunner pid=2049544) step:56 - global_seqlen/min:22164 - global_seqlen/max:48288 - global_seqlen/minmax_diff:26124 - global_seqlen/balanced_min:28371 - global_seqlen/balanced_max:28431 - global_seqlen/mean:28409.375 - actor/entropy:0.21692201495170593 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3174453377723694 - training/rollout_probs_diff_mean:0.003212681272998452 - training/rollout_probs_diff_std:0.010248677805066109 - training/rollout_actor_probs_pearson_corr:0.9989055395126343 - rollout_corr/rollout_is_mean:1.0000497102737427 - rollout_corr/rollout_is_ratio_fraction_high:1.2312617400311865e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.309416181058623e-05 - rollout_corr/rollout_is_max:2.551107883453369 - rollout_corr/rollout_is_min:0.3708029091358185 - rollout_corr/rollout_is_std:0.031356628984212875 - rollout_corr/rollout_is_eff_sample_size:0.9990177275875637 - rollout_corr/rollout_is_seq_mean:1.0000814199447632 - rollout_corr/rollout_is_seq_std:0.002115494105964899 - rollout_corr/rollout_is_seq_max:1.0103763341903687 - rollout_corr/rollout_is_seq_min:0.9925733208656311 - rollout_corr/rollout_is_seq_max_deviation:0.010376334190368652 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2074726885184646) - rollout_corr/training_log_ppl:np.float64(0.18105585960438475) - rollout_corr/kl:np.float64(0.0007089410478542391) - rollout_corr/k3_kl:np.float64(0.0007255429266592728) - rollout_corr/rollout_ppl:np.float64(1.2066095937043428) - rollout_corr/rollout_log_ppl:np.float64(0.18034691788489) - rollout_corr/log_ppl_diff:np.float64(0.00070894171949476) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016144337132573128) - rollout_corr/log_ppl_diff_max:np.float64(0.014080479741096497) - rollout_corr/log_ppl_diff_min:np.float64(-0.00835387408733368) - rollout_corr/ppl_ratio:np.float64(1.0007117404602468) - rollout_corr/chi2_token:np.float64(0.0016396085266023874) - rollout_corr/chi2_seq:np.float64(0.7452940770890564) - actor/pg_loss:np.float64(0.00023605907335877419) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00041431330055274884) - actor/ppo_kl:np.float64(0.00019277200499256963) - actor/pg_clipfrac_lower:np.float64(1.5439723938470706e-05) - actor/grad_norm:np.float64(0.27160290628671646) - perf/mfu/actor:np.float64(0.08149403640203684) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.39385986328125) - actor/lr:np.float64(1e-06) - training/global_step:56 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0020954536739736795 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0020954536739736795 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:634.51171875 - response_length/max:2697.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:634.51171875 - response_length_non_aborted/max:2697.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:253.28125 - prompt_length/max:355.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.834898471832275e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9801836125552654) - timing_s/agent_loop/generate_sequences/max:np.float64(17.489308523014188) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.853927971802477) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.489308523014188) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2659 - timing_s/gen:23.2441786210984 - timing_s/reward:0.08269395306706429 - timing_s/old_log_prob:2.530390776693821 - timing_s/adv:0.136021314188838 - timing_s/update_actor:10.16875221952796 - timing_s/step:36.25585407577455 - timing_s/stop_profile:0.00013040564954280853 - timing_per_token_ms/update_actor:0.04474206234529957 - timing_per_token_ms/gen:0.14309833854217624 - timing_per_token_ms/adv:0.0005984877975529117 - perf/total_num_tokens:227275 - perf/time_per_step:36.25585407577455 - perf/throughput:783.5803547924854 (TaskRunner pid=2049544) Training Progress: 56%|█████▌ | 56/100 [41:59<30:19, 41.35s/it] (TaskRunner pid=2049544) step:57 - global_seqlen/min:25396 - global_seqlen/max:39989 - global_seqlen/minmax_diff:14593 - global_seqlen/balanced_min:32645 - global_seqlen/balanced_max:32684 - global_seqlen/mean:32673.25 - actor/entropy:0.20889751613140106 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.23668746650218964 - training/rollout_probs_diff_mean:0.0030729498248547316 - training/rollout_probs_diff_std:0.009970693849027157 - training/rollout_actor_probs_pearson_corr:0.9990144371986389 - rollout_corr/rollout_is_mean:0.9999489188194275 - rollout_corr/rollout_is_ratio_fraction_high:5.2733157644979656e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.691321035148576e-05 - rollout_corr/rollout_is_max:2.685802698135376 - rollout_corr/rollout_is_min:0.4615940749645233 - rollout_corr/rollout_is_std:0.03016890585422516 - rollout_corr/rollout_is_eff_sample_size:0.9990906647175409 - rollout_corr/rollout_is_seq_mean:0.9998156428337097 - rollout_corr/rollout_is_seq_std:0.0015230864519253373 - rollout_corr/rollout_is_seq_max:1.0045884847640991 - rollout_corr/rollout_is_seq_min:0.9938079118728638 - rollout_corr/rollout_is_seq_max_deviation:0.0061920881271362305 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2169137438759208) - rollout_corr/training_log_ppl:np.float64(0.18850286441738717) - rollout_corr/kl:np.float64(0.0008272747242870082) - rollout_corr/k3_kl:np.float64(0.0005567002334601057) - rollout_corr/rollout_ppl:np.float64(1.2158889258280396) - rollout_corr/rollout_log_ppl:np.float64(0.18767558728723088) - rollout_corr/log_ppl_diff:np.float64(0.0008272771301562898) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012787482482963242) - rollout_corr/log_ppl_diff_max:np.float64(0.00949886441230774) - rollout_corr/log_ppl_diff_min:np.float64(-0.003404974937438965) - rollout_corr/ppl_ratio:np.float64(1.000829064520076) - rollout_corr/chi2_token:np.float64(0.0005316368769854307) - rollout_corr/chi2_seq:np.float64(0.33462699479423463) - actor/pg_loss:np.float64(4.014407750219107e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038230796246807586) - actor/ppo_kl:np.float64(0.00015057841544852124) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21619294956326485) - perf/mfu/actor:np.float64(0.09399730782746046) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.41035079956055) - actor/lr:np.float64(1e-06) - training/global_step:57 - training/epoch:2 - critic/score/mean:0.59765625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.59765625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012718578800559044 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012718578800559044 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:740.7578125 - response_length/max:2402.0 - response_length/min:102.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:740.7578125 - response_length_non_aborted/max:2402.0 - response_length_non_aborted/min:102.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:280.28125 - prompt_length/max:382.0 - prompt_length/min:209.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010364875197410583 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0282678361982107) - timing_s/agent_loop/generate_sequences/max:np.float64(16.842207921668887) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.293985813957988) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.842207921668887) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:361 - timing_s/agent_loop/slowest/response_length:2402 - timing_s/gen:22.68095701187849 - timing_s/reward:0.07416719943284988 - timing_s/old_log_prob:2.519518541172147 - timing_s/adv:0.15157914347946644 - timing_s/update_actor:10.281581053510308 - timing_s/step:35.79420953243971 - timing_s/stop_profile:0.00011889636516571045 - timing_per_token_ms/update_actor:0.03933485746562673 - timing_per_token_ms/gen:0.11960385274728419 - timing_per_token_ms/adv:0.0005799053640189851 - perf/total_num_tokens:261386 - perf/time_per_step:35.79420953243971 - perf/throughput:912.8082566089012 (TaskRunner pid=2049544) Training Progress: 57%|█████▋ | 57/100 [42:34<28:26, 39.69s/it] (TaskRunner pid=2049544) step:58 - global_seqlen/min:19068 - global_seqlen/max:41428 - global_seqlen/minmax_diff:22360 - global_seqlen/balanced_min:32655 - global_seqlen/balanced_max:33260 - global_seqlen/mean:32761.875 - actor/entropy:0.25287628173828125 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.42262715101242065 - training/rollout_probs_diff_mean:0.0032485839910805225 - training/rollout_probs_diff_std:0.009784280322492123 - training/rollout_actor_probs_pearson_corr:0.9992220997810364 - rollout_corr/rollout_is_mean:0.9999250769615173 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.117093729088083e-05 - rollout_corr/rollout_is_max:1.704056739807129 - rollout_corr/rollout_is_min:0.30506160855293274 - rollout_corr/rollout_is_std:0.030359944328665733 - rollout_corr/rollout_is_eff_sample_size:0.9990791225700723 - rollout_corr/rollout_is_seq_mean:0.9998817443847656 - rollout_corr/rollout_is_seq_std:0.0017060940153896809 - rollout_corr/rollout_is_seq_max:1.0051213502883911 - rollout_corr/rollout_is_seq_min:0.9917200803756714 - rollout_corr/rollout_is_seq_max_deviation:0.008279919624328613 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.248608773574233) - rollout_corr/training_log_ppl:np.float64(0.2106571579060983) - rollout_corr/kl:np.float64(0.0006302413064700474) - rollout_corr/k3_kl:np.float64(0.0005439822076880318) - rollout_corr/rollout_ppl:np.float64(1.2478208504617214) - rollout_corr/rollout_log_ppl:np.float64(0.21002691524336115) - rollout_corr/log_ppl_diff:np.float64(0.000630242662737146) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013482350041158497) - rollout_corr/log_ppl_diff_max:np.float64(0.007139943540096283) - rollout_corr/log_ppl_diff_min:np.float64(-0.009232334792613983) - rollout_corr/ppl_ratio:np.float64(1.0006320539396256) - rollout_corr/chi2_token:np.float64(0.0009030492510646582) - rollout_corr/chi2_seq:np.float64(2.417929224204272) - actor/pg_loss:np.float64(-9.389116894453764e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003193014416638107) - actor/ppo_kl:np.float64(-1.695539558355108e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21667541190981865) - perf/mfu/actor:np.float64(0.09071205098155613) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.45818328857422) - actor/lr:np.float64(1e-06) - training/global_step:58 - training/epoch:2 - critic/score/mean:0.5859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.5859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013659363612532616 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013659363612532616 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:751.90234375 - response_length/max:4193.0 - response_length/min:62.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:751.90234375 - response_length_non_aborted/max:4193.0 - response_length_non_aborted/min:62.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.90625 - prompt_length/max:341.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.30633395910263e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7540517617017031) - timing_s/agent_loop/generate_sequences/max:np.float64(25.78281068429351) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.984791398346715) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.78281068429351) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:4193 - timing_s/gen:31.48099230416119 - timing_s/reward:0.07489402964711189 - timing_s/old_log_prob:2.5185533091425896 - timing_s/adv:0.12432781234383583 - timing_s/update_actor:10.674950456246734 - timing_s/step:44.96027087979019 - timing_s/stop_profile:0.00012622401118278503 - timing_per_token_ms/update_actor:0.04072931744690564 - timing_per_token_ms/gen:0.16354866720433686 - timing_per_token_ms/adv:0.00047436163354446225 - perf/total_num_tokens:262095 - perf/time_per_step:44.96027087979019 - perf/throughput:728.6850003104983 (TaskRunner pid=2049544) Training Progress: 58%|█████▊ | 58/100 [43:19<28:54, 41.29s/it] (TaskRunner pid=2049544) step:59 - global_seqlen/min:23178 - global_seqlen/max:36252 - global_seqlen/minmax_diff:13074 - global_seqlen/balanced_min:28566 - global_seqlen/balanced_max:28612 - global_seqlen/mean:28596.625 - actor/entropy:0.24240879714488983 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3797503709793091 - training/rollout_probs_diff_mean:0.0033322961535304785 - training/rollout_probs_diff_std:0.010173420421779156 - training/rollout_actor_probs_pearson_corr:0.9990172982215881 - rollout_corr/rollout_is_mean:1.0001167058944702 - rollout_corr/rollout_is_ratio_fraction_high:6.2438730310532264e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.873162000265438e-05 - rollout_corr/rollout_is_max:2.413045644760132 - rollout_corr/rollout_is_min:0.38240286707878113 - rollout_corr/rollout_is_std:0.03103756159543991 - rollout_corr/rollout_is_eff_sample_size:0.9990379537831949 - rollout_corr/rollout_is_seq_mean:1.0002449750900269 - rollout_corr/rollout_is_seq_std:0.0013697257963940501 - rollout_corr/rollout_is_seq_max:1.003889560699463 - rollout_corr/rollout_is_seq_min:0.9955812692642212 - rollout_corr/rollout_is_seq_max_deviation:0.004418730735778809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2532247374765575) - rollout_corr/training_log_ppl:np.float64(0.21026221416104818) - rollout_corr/kl:np.float64(0.00036410490782223803) - rollout_corr/k3_kl:np.float64(0.0005213920420086282) - rollout_corr/rollout_ppl:np.float64(1.2527254964224994) - rollout_corr/rollout_log_ppl:np.float64(0.20989810848550405) - rollout_corr/log_ppl_diff:np.float64(0.0003641056755441241) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011843454485642724) - rollout_corr/log_ppl_diff_max:np.float64(0.0051978230476379395) - rollout_corr/log_ppl_diff_min:np.float64(-0.004664614796638489) - rollout_corr/ppl_ratio:np.float64(1.000365313142538) - rollout_corr/chi2_token:np.float64(0.0013532442972064018) - rollout_corr/chi2_seq:np.float64(1.167969731381163) - actor/pg_loss:np.float64(5.0825648941099644e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00027953678352332645) - actor/ppo_kl:np.float64(0.0001204940012513589) - actor/pg_clipfrac_lower:np.float64(2.0624340777430916e-06) - actor/grad_norm:np.float64(0.2315312847495079) - perf/mfu/actor:np.float64(0.08222550993023878) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.56365966796875) - actor/lr:np.float64(1e-06) - training/global_step:59 - training/epoch:2 - critic/score/mean:0.62109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.62109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0007726792828179896 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0007726792828179896 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:625.61328125 - response_length/max:2175.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:625.61328125 - response_length_non_aborted/max:2175.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.03125 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.861107587814331e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0195303186774254) - timing_s/agent_loop/generate_sequences/max:np.float64(15.18119196780026) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.312842677085428) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(15.18119196780026) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:274 - timing_s/agent_loop/slowest/response_length:2156 - timing_s/gen:21.4505631364882 - timing_s/reward:0.06381673738360405 - timing_s/old_log_prob:2.4540651980787516 - timing_s/adv:0.12210911512374878 - timing_s/update_actor:10.219576759263873 - timing_s/step:34.34806403145194 - timing_s/stop_profile:0.00012100301682949066 - timing_per_token_ms/update_actor:0.04467125385978185 - timing_per_token_ms/gen:0.13393459628045104 - timing_per_token_ms/adv:0.0005337566720012797 - perf/total_num_tokens:228773 - perf/time_per_step:34.34806403145194 - perf/throughput:832.5542008368959 (TaskRunner pid=2049544) Training Progress: 59%|█████▉ | 59/100 [43:54<26:48, 39.22s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 60} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2 / C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0) \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} \, \mathrm{C} $): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest option is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_50/actor (TaskRunner pid=2049544) step:60 - global_seqlen/min:19702 - global_seqlen/max:41022 - global_seqlen/minmax_diff:21320 - global_seqlen/balanced_min:29482 - global_seqlen/balanced_max:29782 - global_seqlen/mean:29534.0 - actor/entropy:0.2471179962158203 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2885955274105072 - training/rollout_probs_diff_mean:0.0032362197525799274 - training/rollout_probs_diff_std:0.009918652474880219 - training/rollout_actor_probs_pearson_corr:0.9991265535354614 - rollout_corr/rollout_is_mean:1.0000226497650146 - rollout_corr/rollout_is_ratio_fraction_high:1.1968021681241225e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.1968021681241225e-05 - rollout_corr/rollout_is_max:2.9771621227264404 - rollout_corr/rollout_is_min:0.3293769657611847 - rollout_corr/rollout_is_std:0.03091634064912796 - rollout_corr/rollout_is_eff_sample_size:0.999045211617763 - rollout_corr/rollout_is_seq_mean:0.9999569654464722 - rollout_corr/rollout_is_seq_std:0.0014956871746107936 - rollout_corr/rollout_is_seq_max:1.0054192543029785 - rollout_corr/rollout_is_seq_min:0.9950917959213257 - rollout_corr/rollout_is_seq_max_deviation:0.005419254302978516 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2401416734792292) - rollout_corr/training_log_ppl:np.float64(0.19959818758798065) - rollout_corr/kl:np.float64(0.0005349192640053957) - rollout_corr/k3_kl:np.float64(0.0004940067541099324) - rollout_corr/rollout_ppl:np.float64(1.2394655891694129) - rollout_corr/rollout_log_ppl:np.float64(0.19906326734781032) - rollout_corr/log_ppl_diff:np.float64(0.0005349202401703224) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011775598395615816) - rollout_corr/log_ppl_diff_max:np.float64(0.005937367677688599) - rollout_corr/log_ppl_diff_min:np.float64(-0.004511777311563492) - rollout_corr/ppl_ratio:np.float64(1.000536180799827) - rollout_corr/chi2_token:np.float64(0.0009251013398170471) - rollout_corr/chi2_seq:np.float64(1.390581222018227) - actor/pg_loss:np.float64(4.893657751381397e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018130719581677113) - actor/ppo_kl:np.float64(-2.2069814953340483e-07) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.15835927985608578) - perf/mfu/actor:np.float64(0.08403507336330182) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.51084518432617) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6375) - val-aux/sciknoweval/reward/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10267672443906181) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8312625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.052249733074818304) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6821875000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.076862250314467) - val-aux/sciknoweval/score/mean@16:np.float64(0.6375) - val-aux/sciknoweval/score/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/score/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/score/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/score/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10267672443906181) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8312625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.052249733074818304) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6821875000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.076862250314467) - val-core/sciknoweval/acc/mean@16:np.float64(0.6375) - val-aux/sciknoweval/acc/std@16:np.float64(0.1808793393521641) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.71125) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.14531694283181013) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.5644) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15827951046753036) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6389125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.18130674840602382) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.766225) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.1033040821613348) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.49958749999999996) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12475642320545952) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6604125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1416569094891499) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8033625000000001) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.07008229463050868) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.449375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08561471923061226) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6742) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10267672443906181) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8312625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04984537510833955) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.41555) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.052249733074818304) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6821875000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.076862250314467) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9765625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.0574668098874074) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9954875000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.022722407353494737) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9575250000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.07239005753165481) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9770249999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.05686531821374918) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9995499999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.004893080696751963) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9285500000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.08215960170640288) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9901500000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03655386464803364) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(0.9999750000000001) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0005584576975922169) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8929375) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08171927324120949) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9971125000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.017114100398696087) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8558125000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.06612062868202828) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9996750000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.00411263922012108) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:60 - training/epoch:2 - critic/score/mean:0.6484375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6484375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00966417696326971 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.00966417696326971 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:652.78125 - response_length/max:3139.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:652.78125 - response_length_non_aborted/max:3139.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.15625 - prompt_length/max:340.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.66247171163559e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2835856825113297) - timing_s/agent_loop/generate_sequences/max:np.float64(20.483323162421584) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.35434487216844) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.483323162421584) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:197 - timing_s/agent_loop/slowest/response_length:3139 - timing_s/gen:26.403195202350616 - timing_s/reward:0.06402513571083546 - timing_s/old_log_prob:2.5449625328183174 - timing_s/adv:0.11957781203091145 - timing_s/update_actor:10.040485488250852 - timing_s/step:39.238921258598566 - timing_s/testing:114.50520241633058 - timing_s/save_checkpoint:6.870979869738221 - timing_s/stop_profile:0.00015047751367092133 - timing_per_token_ms/update_actor:0.04249545222561646 - timing_per_token_ms/gen:0.15799700322149587 - timing_per_token_ms/adv:0.0005061023398071351 - perf/total_num_tokens:236272 - perf/time_per_step:39.238921258598566 - perf/throughput:752.6710483542691 (TaskRunner pid=2049544) Training Progress: 60%|██████ | 60/100 [46:34<50:26, 75.65s/it] (TaskRunner pid=2049544) step:61 - global_seqlen/min:22238 - global_seqlen/max:39486 - global_seqlen/minmax_diff:17248 - global_seqlen/balanced_min:29924 - global_seqlen/balanced_max:30089 - global_seqlen/mean:29970.875 - actor/entropy:0.23904968798160553 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5477294921875 - training/rollout_probs_diff_mean:0.003164143767207861 - training/rollout_probs_diff_std:0.00985527690500021 - training/rollout_actor_probs_pearson_corr:0.9991676211357117 - rollout_corr/rollout_is_mean:1.0000258684158325 - rollout_corr/rollout_is_ratio_fraction_high:1.1471639481897e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.720745967759285e-05 - rollout_corr/rollout_is_max:4.02189826965332 - rollout_corr/rollout_is_min:0.26447275280952454 - rollout_corr/rollout_is_std:0.030506834387779236 - rollout_corr/rollout_is_eff_sample_size:0.9990700794243639 - rollout_corr/rollout_is_seq_mean:1.0000176429748535 - rollout_corr/rollout_is_seq_std:0.0016575142508372664 - rollout_corr/rollout_is_seq_max:1.0063189268112183 - rollout_corr/rollout_is_seq_min:0.9920255541801453 - rollout_corr/rollout_is_seq_max_deviation:0.007974445819854736 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.25040641753003) - rollout_corr/training_log_ppl:np.float64(0.20799311240261886) - rollout_corr/kl:np.float64(0.0005623800304297433) - rollout_corr/k3_kl:np.float64(0.0005192289710862497) - rollout_corr/rollout_ppl:np.float64(1.2496482259593904) - rollout_corr/rollout_log_ppl:np.float64(0.20743073029007064) - rollout_corr/log_ppl_diff:np.float64(0.0005623821125482209) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012774796996382065) - rollout_corr/log_ppl_diff_max:np.float64(0.009329631924629211) - rollout_corr/log_ppl_diff_min:np.float64(-0.007050305604934692) - rollout_corr/ppl_ratio:np.float64(1.0005640257149935) - rollout_corr/chi2_token:np.float64(0.0009777885861694813) - rollout_corr/chi2_seq:np.float64(0.3470004736445844) - actor/pg_loss:np.float64(-0.0002099055564031005) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002631939267985217) - actor/ppo_kl:np.float64(6.142829495781044e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.23834596201777458) - perf/mfu/actor:np.float64(0.08600884737044563) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.00771045684814) - actor/lr:np.float64(1e-06) - training/global_step:61 - training/epoch:2 - critic/score/mean:0.68359375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.68359375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007454414386302233 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:0.007454414386302233 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:681.02734375 - response_length/max:3128.0 - response_length/min:77.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:681.02734375 - response_length_non_aborted/max:3128.0 - response_length_non_aborted/min:77.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:255.5625 - prompt_length/max:328.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00015644170343875885 - timing_s/agent_loop/generate_sequences/min:np.float64(0.93056246265769) - timing_s/agent_loop/generate_sequences/max:np.float64(20.83841021731496) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.5293213194963755) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.83841021731496) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:246 - timing_s/agent_loop/slowest/response_length:3128 - timing_s/gen:26.53979305177927 - timing_s/reward:0.08145093731582165 - timing_s/old_log_prob:2.588765997439623 - timing_s/adv:0.11384812742471695 - timing_s/update_actor:10.279441229999065 - timing_s/step:39.70467999763787 - timing_s/stop_profile:0.0001134183257818222 - timing_per_token_ms/update_actor:0.04287262730066717 - timing_per_token_ms/gen:0.1522274656956647 - timing_per_token_ms/adv:0.0004748281766244602 - perf/total_num_tokens:239767 - perf/time_per_step:39.70467999763787 - perf/throughput:754.8448949036497 (TaskRunner pid=2049544) Training Progress: 61%|██████ | 61/100 [47:14<42:10, 64.88s/it] (TaskRunner pid=2049544) step:62 - global_seqlen/min:24330 - global_seqlen/max:43162 - global_seqlen/minmax_diff:18832 - global_seqlen/balanced_min:34742 - global_seqlen/balanced_max:34794 - global_seqlen/mean:34771.375 - actor/entropy:0.2732798457145691 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4058573246002197 - training/rollout_probs_diff_mean:0.00327512389048934 - training/rollout_probs_diff_std:0.009542555548250675 - training/rollout_actor_probs_pearson_corr:0.9991864562034607 - rollout_corr/rollout_is_mean:0.9999688267707825 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.467516540287761e-05 - rollout_corr/rollout_is_max:1.9362671375274658 - rollout_corr/rollout_is_min:0.37468504905700684 - rollout_corr/rollout_is_std:0.030330481007695198 - rollout_corr/rollout_is_eff_sample_size:0.9990807884305588 - rollout_corr/rollout_is_seq_mean:0.9999213814735413 - rollout_corr/rollout_is_seq_std:0.0014323026407510042 - rollout_corr/rollout_is_seq_max:1.0065138339996338 - rollout_corr/rollout_is_seq_min:0.9906108975410461 - rollout_corr/rollout_is_seq_max_deviation:0.009389102458953857 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2884436114691198) - rollout_corr/training_log_ppl:np.float64(0.23855835630820366) - rollout_corr/kl:np.float64(0.0005924100263499099) - rollout_corr/k3_kl:np.float64(0.0005057925414178044) - rollout_corr/rollout_ppl:np.float64(1.2876463397406042) - rollout_corr/rollout_log_ppl:np.float64(0.23796594546729466) - rollout_corr/log_ppl_diff:np.float64(0.0005924108409089968) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011562938307179138) - rollout_corr/log_ppl_diff_max:np.float64(0.009115807712078094) - rollout_corr/log_ppl_diff_min:np.float64(-0.0043695345520973206) - rollout_corr/ppl_ratio:np.float64(1.0005937188398093) - rollout_corr/chi2_token:np.float64(0.0008554272353649139) - rollout_corr/chi2_seq:np.float64(0.8644126462750137) - actor/pg_loss:np.float64(-5.7160970754921436e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00029845362780633877) - actor/ppo_kl:np.float64(3.140159503445261e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22896912693977356) - perf/mfu/actor:np.float64(0.09839975519482527) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0562973022461) - actor/lr:np.float64(1e-06) - training/global_step:62 - training/epoch:2 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.022515371441841125 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.022515371441841125 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:798.54296875 - response_length/max:3135.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:798.54296875 - response_length_non_aborted/max:3135.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:288.0625 - prompt_length/max:460.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00014179199934005737 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8601270448416471) - timing_s/agent_loop/generate_sequences/max:np.float64(20.932036239653826) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.75760180906218) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.932036239653826) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:460 - timing_s/agent_loop/slowest/response_length:3135 - timing_s/gen:27.040857832878828 - timing_s/reward:0.07787876017391682 - timing_s/old_log_prob:2.5947242341935635 - timing_s/adv:0.12395207956433296 - timing_s/update_actor:10.449679136276245 - timing_s/step:40.37802067771554 - timing_s/stop_profile:0.0001217275857925415 - timing_per_token_ms/update_actor:0.03756566693248486 - timing_per_token_ms/gen:0.1322763521104298 - timing_per_token_ms/adv:0.0004455966997434418 - perf/total_num_tokens:278171 - perf/time_per_step:40.37802067771554 - perf/throughput:861.1460991992154 (TaskRunner pid=2049544) Training Progress: 62%|██████▏ | 62/100 [47:55<36:26, 57.55s/it] (TaskRunner pid=2049544) step:63 - global_seqlen/min:21359 - global_seqlen/max:35870 - global_seqlen/minmax_diff:14511 - global_seqlen/balanced_min:26722 - global_seqlen/balanced_max:27254 - global_seqlen/mean:26812.0 - actor/entropy:0.27676644921302795 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.9831006526947021 - training/rollout_probs_diff_mean:0.0035199327394366264 - training/rollout_probs_diff_std:0.010530827566981316 - training/rollout_actor_probs_pearson_corr:0.9990900158882141 - rollout_corr/rollout_is_mean:0.9999789595603943 - rollout_corr/rollout_is_ratio_fraction_high:1.3679142284672707e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.419785571168177e-05 - rollout_corr/rollout_is_max:2.8213930130004883 - rollout_corr/rollout_is_min:0.011052249930799007 - rollout_corr/rollout_is_std:0.031423088163137436 - rollout_corr/rollout_is_eff_sample_size:0.9990135634724805 - rollout_corr/rollout_is_seq_mean:1.0000073909759521 - rollout_corr/rollout_is_seq_std:0.001668434706516564 - rollout_corr/rollout_is_seq_max:1.0121815204620361 - rollout_corr/rollout_is_seq_min:0.9929383993148804 - rollout_corr/rollout_is_seq_max_deviation:0.012181520462036133 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2705106027424335) - rollout_corr/training_log_ppl:np.float64(0.22211689058167394) - rollout_corr/kl:np.float64(0.0004739185238458177) - rollout_corr/k3_kl:np.float64(0.0005098591802266128) - rollout_corr/rollout_ppl:np.float64(1.2698905346915126) - rollout_corr/rollout_log_ppl:np.float64(0.2216429725958733) - rollout_corr/log_ppl_diff:np.float64(0.0004739179858006537) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012847277976106852) - rollout_corr/log_ppl_diff_max:np.float64(0.0076948776841163635) - rollout_corr/log_ppl_diff_min:np.float64(-0.010769776999950409) - rollout_corr/ppl_ratio:np.float64(1.0004755130503327) - rollout_corr/chi2_token:np.float64(0.001110320445150137) - rollout_corr/chi2_seq:np.float64(0.7298937705345452) - actor/pg_loss:np.float64(-6.4569758251309395e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017891856134610862) - actor/ppo_kl:np.float64(-4.981012998683809e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.19124767929315567) - perf/mfu/actor:np.float64(0.07698739538780973) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.1104507446289) - actor/lr:np.float64(1e-06) - training/global_step:63 - training/epoch:2 - critic/score/mean:0.6640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018338599475100636 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:-0.0018338599475100636 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:571.125 - response_length/max:3010.0 - response_length/min:78.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:571.125 - response_length_non_aborted/max:3010.0 - response_length_non_aborted/min:78.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.75 - prompt_length/max:350.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001445487141609192 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8224132973700762) - timing_s/agent_loop/generate_sequences/max:np.float64(18.348940243944526) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.530061706544075) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.348940243944526) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:268 - timing_s/agent_loop/slowest/response_length:3010 - timing_s/gen:25.11938327550888 - timing_s/reward:0.06868074834346771 - timing_s/old_log_prob:2.4906132221221924 - timing_s/adv:0.12919375859200954 - timing_s/update_actor:10.244162807241082 - timing_s/step:38.14616190828383 - timing_s/stop_profile:0.00011005811393260956 - timing_per_token_ms/update_actor:0.047759225380618206 - timing_per_token_ms/gen:0.17180580594433192 - timing_per_token_ms/adv:0.0006023131368044604 - perf/total_num_tokens:214496 - perf/time_per_step:38.14616190828383 - perf/throughput:702.8754312023591 (TaskRunner pid=2049544) Training Progress: 63%|██████▎ | 63/100 [48:33<31:54, 51.74s/it] (TaskRunner pid=2049544) step:64 - global_seqlen/min:26498 - global_seqlen/max:40322 - global_seqlen/minmax_diff:13824 - global_seqlen/balanced_min:31563 - global_seqlen/balanced_max:31618 - global_seqlen/mean:31602.875 - actor/entropy:0.28700023889541626 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3535178303718567 - training/rollout_probs_diff_mean:0.0033518956042826176 - training/rollout_probs_diff_std:0.009729590266942978 - training/rollout_actor_probs_pearson_corr:0.9992855787277222 - rollout_corr/rollout_is_mean:1.0000327825546265 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.0833825399458874e-05 - rollout_corr/rollout_is_max:1.9838535785675049 - rollout_corr/rollout_is_min:0.4158313572406769 - rollout_corr/rollout_is_std:0.03056929260492325 - rollout_corr/rollout_is_eff_sample_size:0.9990665098066491 - rollout_corr/rollout_is_seq_mean:0.9999979734420776 - rollout_corr/rollout_is_seq_std:0.0013619516976177692 - rollout_corr/rollout_is_seq_max:1.0039030313491821 - rollout_corr/rollout_is_seq_min:0.9954256415367126 - rollout_corr/rollout_is_seq_max_deviation:0.0045743584632873535 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2764493175782263) - rollout_corr/training_log_ppl:np.float64(0.22693806317693088) - rollout_corr/kl:np.float64(0.00046152420509582015) - rollout_corr/k3_kl:np.float64(0.0005074531937054871) - rollout_corr/rollout_ppl:np.float64(1.275848121382296) - rollout_corr/rollout_log_ppl:np.float64(0.22647653929016087) - rollout_corr/log_ppl_diff:np.float64(0.00046152388677001) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011273344280198216) - rollout_corr/log_ppl_diff_max:np.float64(0.00530514121055603) - rollout_corr/log_ppl_diff_min:np.float64(-0.005483843386173248) - rollout_corr/ppl_ratio:np.float64(1.0004626805894077) - rollout_corr/chi2_token:np.float64(0.0011098254472017288) - rollout_corr/chi2_seq:np.float64(1.8452809148002416) - actor/pg_loss:np.float64(-0.00011544267181307077) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002127377759961746) - actor/ppo_kl:np.float64(-4.476623224114462e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18739718943834305) - perf/mfu/actor:np.float64(0.08994342294998352) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.98906707763672) - actor/lr:np.float64(1e-06) - training/global_step:64 - training/epoch:2 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.013745415955781937 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.013745415955781937 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:721.12109375 - response_length/max:2725.0 - response_length/min:175.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:721.12109375 - response_length_non_aborted/max:2725.0 - response_length_non_aborted/min:175.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.46875 - prompt_length/max:349.0 - prompt_length/min:192.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.722448885440826e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.080433465540409) - timing_s/agent_loop/generate_sequences/max:np.float64(17.78822729177773) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.979315571334155) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.78822729177773) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:2560 - timing_s/gen:23.619467133656144 - timing_s/reward:0.07297545671463013 - timing_s/old_log_prob:2.6043873876333237 - timing_s/adv:0.14066699519753456 - timing_s/update_actor:10.391787450760603 - timing_s/step:36.91719730384648 - timing_s/stop_profile:0.0001245010644197464 - timing_per_token_ms/update_actor:0.041103014562601514 - timing_per_token_ms/gen:0.127944591124151 - timing_per_token_ms/adv:0.0005563852782283833 - perf/total_num_tokens:252823 - perf/time_per_step:36.91719730384648 - perf/throughput:856.047514655378 (TaskRunner pid=2049544) Training Progress: 64%|██████▍ | 64/100 [49:10<28:23, 47.32s/it] (TaskRunner pid=2049544) step:65 - global_seqlen/min:13817 - global_seqlen/max:44402 - global_seqlen/minmax_diff:30585 - global_seqlen/balanced_min:27087 - global_seqlen/balanced_max:27398 - global_seqlen/mean:27214.25 - actor/entropy:0.2703191936016083 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.27730000019073486 - training/rollout_probs_diff_mean:0.003337751142680645 - training/rollout_probs_diff_std:0.009712927974760532 - training/rollout_actor_probs_pearson_corr:0.999212384223938 - rollout_corr/rollout_is_mean:0.9999141097068787 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3306897017173469e-05 - rollout_corr/rollout_is_max:1.8171900510787964 - rollout_corr/rollout_is_min:0.37625378370285034 - rollout_corr/rollout_is_std:0.030340304598212242 - rollout_corr/rollout_is_eff_sample_size:0.999079955499621 - rollout_corr/rollout_is_seq_mean:1.0000232458114624 - rollout_corr/rollout_is_seq_std:0.001954729435965419 - rollout_corr/rollout_is_seq_max:1.0170458555221558 - rollout_corr/rollout_is_seq_min:0.9947062730789185 - rollout_corr/rollout_is_seq_max_deviation:0.01704585552215576 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2311511896550655) - rollout_corr/training_log_ppl:np.float64(0.1959650574644911) - rollout_corr/kl:np.float64(0.0005068062779431948) - rollout_corr/k3_kl:np.float64(0.0005566824616494159) - rollout_corr/rollout_ppl:np.float64(1.2305298056453466) - rollout_corr/rollout_log_ppl:np.float64(0.1954582512989873) - rollout_corr/log_ppl_diff:np.float64(0.0005068061655038036) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013243403445812874) - rollout_corr/log_ppl_diff_max:np.float64(0.007357165217399597) - rollout_corr/log_ppl_diff_min:np.float64(-0.020653128623962402) - rollout_corr/ppl_ratio:np.float64(1.0005090332124382) - rollout_corr/chi2_token:np.float64(0.0013379347510635853) - rollout_corr/chi2_seq:np.float64(0.5852078427560627) - actor/pg_loss:np.float64(5.363975651562214e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001393496023638363) - actor/ppo_kl:np.float64(4.77582690940892e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17264313623309135) - perf/mfu/actor:np.float64(0.07762399981838528) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82825469970703) - actor/lr:np.float64(1e-06) - training/global_step:65 - training/epoch:2 - critic/score/mean:0.890625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.890625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0034032389521598816 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:0.0034032389521598816 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:587.1015625 - response_length/max:3738.0 - response_length/min:88.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:587.1015625 - response_length_non_aborted/max:3738.0 - response_length_non_aborted/min:88.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.34375 - prompt_length/max:364.0 - prompt_length/min:176.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.999820470809937e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0192465409636497) - timing_s/agent_loop/generate_sequences/max:np.float64(20.993049459531903) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.240891806744912) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.993049459531903) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:3738 - timing_s/gen:26.803196189925075 - timing_s/reward:0.06635245122015476 - timing_s/old_log_prob:2.5201703682541847 - timing_s/adv:0.12346276640892029 - timing_s/update_actor:10.165898667648435 - timing_s/step:39.771548522636294 - timing_s/stop_profile:0.00011387281119823456 - timing_per_token_ms/update_actor:0.04669382156245549 - timing_per_token_ms/gen:0.17833368501194344 - timing_per_token_ms/adv:0.0005670869416248853 - perf/total_num_tokens:217714 - perf/time_per_step:39.771548522636294 - perf/throughput:684.2642796397729 (TaskRunner pid=2049544) Training Progress: 65%|██████▌ | 65/100 [49:50<26:17, 45.07s/it] (TaskRunner pid=2049544) step:66 - global_seqlen/min:26398 - global_seqlen/max:47072 - global_seqlen/minmax_diff:20674 - global_seqlen/balanced_min:34031 - global_seqlen/balanced_max:34100 - global_seqlen/mean:34083.75 - actor/entropy:0.32043859362602234 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4526241421699524 - training/rollout_probs_diff_mean:0.003443863708525896 - training/rollout_probs_diff_std:0.009328007698059082 - training/rollout_actor_probs_pearson_corr:0.999345064163208 - rollout_corr/rollout_is_mean:0.999977171421051 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.9896735466318205e-05 - rollout_corr/rollout_is_max:1.9577182531356812 - rollout_corr/rollout_is_min:0.13161711394786835 - rollout_corr/rollout_is_std:0.030255712568759918 - rollout_corr/rollout_is_eff_sample_size:0.9990853100798829 - rollout_corr/rollout_is_seq_mean:1.0000357627868652 - rollout_corr/rollout_is_seq_std:0.0014041649410501122 - rollout_corr/rollout_is_seq_max:1.0082844495773315 - rollout_corr/rollout_is_seq_min:0.9953852295875549 - rollout_corr/rollout_is_seq_max_deviation:0.008284449577331543 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3104135780595243) - rollout_corr/training_log_ppl:np.float64(0.25584780907956883) - rollout_corr/kl:np.float64(0.0005620165548441136) - rollout_corr/k3_kl:np.float64(0.0004995921321437891) - rollout_corr/rollout_ppl:np.float64(1.309668519999832) - rollout_corr/rollout_log_ppl:np.float64(0.2552857908885926) - rollout_corr/log_ppl_diff:np.float64(0.0005620181909762323) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011583589657675475) - rollout_corr/log_ppl_diff_max:np.float64(0.005684643983840942) - rollout_corr/log_ppl_diff_min:np.float64(-0.006929904222488403) - rollout_corr/ppl_ratio:np.float64(1.000563226873055) - rollout_corr/chi2_token:np.float64(0.0008636738639324903) - rollout_corr/chi2_seq:np.float64(0.8563104090280831) - actor/pg_loss:np.float64(0.0001647728495299816) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017283655381561402) - actor/ppo_kl:np.float64(0.00013175434309165013) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1924263872206211) - perf/mfu/actor:np.float64(0.09599912196819169) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.91796875) - actor/lr:np.float64(1e-06) - training/global_step:66 - training/epoch:2 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.000762293697334826 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.000762293697334826 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:785.3046875 - response_length/max:3216.0 - response_length/min:108.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:785.3046875 - response_length_non_aborted/max:3216.0 - response_length_non_aborted/min:108.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.8125 - prompt_length/max:365.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.582083344459534e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2348940391093493) - timing_s/agent_loop/generate_sequences/max:np.float64(21.131182158365846) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.555415921531676) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.131182158365846) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3216 - timing_s/gen:26.812409596517682 - timing_s/reward:0.11229918338358402 - timing_s/old_log_prob:2.537730697542429 - timing_s/adv:0.12344273924827576 - timing_s/update_actor:10.477383820340037 - timing_s/step:40.15382357686758 - timing_s/stop_profile:0.00011853314936161041 - timing_per_token_ms/update_actor:0.03842514328800395 - timing_per_token_ms/gen:0.1333698584174021 - timing_per_token_ms/adv:0.00045271844811778253 - perf/total_num_tokens:272670 - perf/time_per_step:40.15382357686758 - perf/throughput:848.8295002530091 (TaskRunner pid=2049544) Training Progress: 66%|██████▌ | 66/100 [50:30<24:42, 43.61s/it] (TaskRunner pid=2049544) step:67 - global_seqlen/min:14820 - global_seqlen/max:39555 - global_seqlen/minmax_diff:24735 - global_seqlen/balanced_min:25556 - global_seqlen/balanced_max:26212 - global_seqlen/mean:25672.375 - actor/entropy:0.27359628677368164 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.46210977435112 - training/rollout_probs_diff_mean:0.0033997194841504097 - training/rollout_probs_diff_std:0.009928795509040356 - training/rollout_actor_probs_pearson_corr:0.9991419315338135 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.1833584469277412e-05 - rollout_corr/rollout_is_max:1.9785306453704834 - rollout_corr/rollout_is_min:0.3678695261478424 - rollout_corr/rollout_is_std:0.031190816313028336 - rollout_corr/rollout_is_eff_sample_size:0.9990281975159966 - rollout_corr/rollout_is_seq_mean:1.0001834630966187 - rollout_corr/rollout_is_seq_std:0.002221266273409128 - rollout_corr/rollout_is_seq_max:1.0159006118774414 - rollout_corr/rollout_is_seq_min:0.9894429445266724 - rollout_corr/rollout_is_seq_max_deviation:0.015900611877441406 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3023830708116293) - rollout_corr/training_log_ppl:np.float64(0.2422099069954129) - rollout_corr/kl:np.float64(0.00026683149832429365) - rollout_corr/k3_kl:np.float64(0.000541509634160775) - rollout_corr/rollout_ppl:np.float64(1.301996021065861) - rollout_corr/rollout_log_ppl:np.float64(0.241943074419396) - rollout_corr/log_ppl_diff:np.float64(0.0002668325760168955) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014319048641482368) - rollout_corr/log_ppl_diff_max:np.float64(0.009274959564208984) - rollout_corr/log_ppl_diff_min:np.float64(-0.0138968825340271) - rollout_corr/ppl_ratio:np.float64(1.0002689892426133) - rollout_corr/chi2_token:np.float64(0.0016502379439771175) - rollout_corr/chi2_seq:np.float64(1.019609775627032) - actor/pg_loss:np.float64(-5.0529371947050095e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016554813828406623) - actor/ppo_kl:np.float64(-7.827897409384832e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17443475499749184) - perf/mfu/actor:np.float64(0.07407053273139065) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82710266113281) - actor/lr:np.float64(1e-06) - training/global_step:67 - training/epoch:3 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01963839866220951 - critic/advantages/max:0.625 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01963839866220951 - critic/returns/max:0.625 - critic/returns/min:-0.875 - response_length/mean:536.73046875 - response_length/max:3340.0 - response_length/min:85.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:536.73046875 - response_length_non_aborted/max:3340.0 - response_length_non_aborted/min:85.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.53125 - prompt_length/max:361.0 - prompt_length/min:172.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00023074820637702942 - timing_s/agent_loop/generate_sequences/min:np.float64(1.033557141199708) - timing_s/agent_loop/generate_sequences/max:np.float64(19.718276670202613) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.309584286827885) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.718276670202613) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:3340 - timing_s/gen:25.747549824416637 - timing_s/reward:0.05820099823176861 - timing_s/old_log_prob:2.587189307436347 - timing_s/adv:0.1540521327406168 - timing_s/update_actor:10.0051255710423 - timing_s/step:38.650182999670506 - timing_s/stop_profile:0.0001270286738872528 - timing_per_token_ms/update_actor:0.04871542646055487 - timing_per_token_ms/gen:0.18738710089602584 - timing_per_token_ms/adv:0.00075008707190422 - perf/total_num_tokens:205379 - perf/time_per_step:38.650182999670506 - perf/throughput:664.2238925548905 (TaskRunner pid=2049544) Training Progress: 67%|██████▋ | 67/100 [51:09<23:14, 42.26s/it] (TaskRunner pid=2049544) step:68 - global_seqlen/min:21420 - global_seqlen/max:36370 - global_seqlen/minmax_diff:14950 - global_seqlen/balanced_min:26998 - global_seqlen/balanced_max:27019 - global_seqlen/mean:27007.25 - actor/entropy:0.2508779466152191 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4620983600616455 - training/rollout_probs_diff_mean:0.003281368175521493 - training/rollout_probs_diff_std:0.009971853345632553 - training/rollout_actor_probs_pearson_corr:0.9991183876991272 - rollout_corr/rollout_is_mean:0.9999003410339355 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:3.37541350745596e-05 - rollout_corr/rollout_is_max:1.727583885192871 - rollout_corr/rollout_is_min:0.3678779602050781 - rollout_corr/rollout_is_std:0.03009769693017006 - rollout_corr/rollout_is_eff_sample_size:0.9990948294754013 - rollout_corr/rollout_is_seq_mean:0.999991238117218 - rollout_corr/rollout_is_seq_std:0.0015605590306222439 - rollout_corr/rollout_is_seq_max:1.005854606628418 - rollout_corr/rollout_is_seq_min:0.9957244396209717 - rollout_corr/rollout_is_seq_max_deviation:0.005854606628417969 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2427316303364933) - rollout_corr/training_log_ppl:np.float64(0.2052579416122171) - rollout_corr/kl:np.float64(0.0006245356830429927) - rollout_corr/k3_kl:np.float64(0.0004809370777678623) - rollout_corr/rollout_ppl:np.float64(1.2419258821755648) - rollout_corr/rollout_log_ppl:np.float64(0.20463340327114565) - rollout_corr/log_ppl_diff:np.float64(0.0006245383410714567) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013257119280751795) - rollout_corr/log_ppl_diff_max:np.float64(0.005434125661849976) - rollout_corr/log_ppl_diff_min:np.float64(-0.0038042664527893066) - rollout_corr/ppl_ratio:np.float64(1.0006259840447456) - rollout_corr/chi2_token:np.float64(0.0006594203878194094) - rollout_corr/chi2_seq:np.float64(0.44312575249932706) - actor/pg_loss:np.float64(-5.7110912166535854e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002576871158908034) - actor/ppo_kl:np.float64(0.000146509498037517) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1650708168745041) - perf/mfu/actor:np.float64(0.07864794728282518) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82149505615234) - actor/lr:np.float64(1e-06) - training/global_step:68 - training/epoch:3 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00015104975318536162 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00015104975318536162 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:578.6328125 - response_length/max:1942.0 - response_length/min:89.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:578.6328125 - response_length_non_aborted/max:1942.0 - response_length_non_aborted/min:89.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:265.34375 - prompt_length/max:345.0 - prompt_length/min:161.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.873881936073303e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.0787797346711159) - timing_s/agent_loop/generate_sequences/max:np.float64(13.899392530322075) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.687431053593173) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.899392530322075) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:341 - timing_s/agent_loop/slowest/response_length:1942 - timing_s/gen:19.621694235131145 - timing_s/reward:0.06307606771588326 - timing_s/old_log_prob:2.414205303415656 - timing_s/adv:0.1221406627446413 - timing_s/update_actor:10.004864446818829 - timing_s/step:32.31770410388708 - timing_s/stop_profile:4.9777328968048096e-05 - timing_per_token_ms/update_actor:0.04630638276212327 - timing_per_token_ms/gen:0.13246266276332375 - timing_per_token_ms/adv:0.0005653142338846111 - perf/total_num_tokens:216058 - perf/time_per_step:32.31770410388708 - perf/throughput:835.6797225812722 (TaskRunner pid=2049544) Training Progress: 68%|██████▊ | 68/100 [51:41<20:57, 39.29s/it] (TaskRunner pid=2049544) step:69 - global_seqlen/min:23146 - global_seqlen/max:36753 - global_seqlen/minmax_diff:13607 - global_seqlen/balanced_min:29343 - global_seqlen/balanced_max:29374 - global_seqlen/mean:29363.125 - actor/entropy:0.2744224965572357 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24491748213768005 - training/rollout_probs_diff_mean:0.0033163181506097317 - training/rollout_probs_diff_std:0.00957618560642004 - training/rollout_actor_probs_pearson_corr:0.9992418885231018 - rollout_corr/rollout_is_mean:0.9999679327011108 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:1.9769123792648315 - rollout_corr/rollout_is_min:0.547244668006897 - rollout_corr/rollout_is_std:0.030115514993667603 - rollout_corr/rollout_is_eff_sample_size:0.9990938775276863 - rollout_corr/rollout_is_seq_mean:1.000019907951355 - rollout_corr/rollout_is_seq_std:0.001533227856270969 - rollout_corr/rollout_is_seq_max:1.0057690143585205 - rollout_corr/rollout_is_seq_min:0.990231990814209 - rollout_corr/rollout_is_seq_max_deviation:0.009768009185791016 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2841168828308582) - rollout_corr/training_log_ppl:np.float64(0.23235505366756115) - rollout_corr/kl:np.float64(0.0005125461427646361) - rollout_corr/k3_kl:np.float64(0.0005065118215270559) - rollout_corr/rollout_ppl:np.float64(1.2834058650769293) - rollout_corr/rollout_log_ppl:np.float64(0.2318425054399995) - rollout_corr/log_ppl_diff:np.float64(0.0005125482275616378) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012335704232100397) - rollout_corr/log_ppl_diff_max:np.float64(0.011537343263626099) - rollout_corr/log_ppl_diff_min:np.float64(-0.005053341388702393) - rollout_corr/ppl_ratio:np.float64(1.0005140437278897) - rollout_corr/chi2_token:np.float64(0.00102224200963974) - rollout_corr/chi2_seq:np.float64(0.8634938539471477) - actor/pg_loss:np.float64(-5.3855590522289276e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000181908647164164) - actor/ppo_kl:np.float64(5.1798504737021744e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18398911505937576) - perf/mfu/actor:np.float64(0.08486034559374442) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.82257843017578) - actor/lr:np.float64(1e-06) - training/global_step:69 - training/epoch:3 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.015627924352884293 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.015627924352884293 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:646.87890625 - response_length/max:2353.0 - response_length/min:105.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:646.87890625 - response_length_non_aborted/max:2353.0 - response_length_non_aborted/min:105.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:363.0 - prompt_length/min:179.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:4.917755722999573e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.203237522393465) - timing_s/agent_loop/generate_sequences/max:np.float64(16.094254648312926) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.346816844648856) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.094254648312926) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:260 - timing_s/agent_loop/slowest/response_length:2353 - timing_s/gen:22.094875816255808 - timing_s/reward:0.05950486660003662 - timing_s/old_log_prob:2.4449626710265875 - timing_s/adv:0.12635387293994427 - timing_s/update_actor:10.193446822464466 - timing_s/step:34.9527602866292 - timing_s/stop_profile:2.9163435101509094e-05 - timing_per_token_ms/update_actor:0.04339391167690967 - timing_per_token_ms/gen:0.13342235745107703 - timing_per_token_ms/adv:0.0005378935013726581 - perf/total_num_tokens:234905 - perf/time_per_step:34.9527602866292 - perf/throughput:840.0802900603117 (TaskRunner pid=2049544) Training Progress: 69%|██████▉ | 69/100 [52:16<19:37, 37.99s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 70} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E \cdot r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \cdot (0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) = \frac{2.0 \cdot 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_60/actor (TaskRunner pid=2049544) step:70 - global_seqlen/min:18463 - global_seqlen/max:39933 - global_seqlen/minmax_diff:21470 - global_seqlen/balanced_min:27119 - global_seqlen/balanced_max:27177 - global_seqlen/mean:27165.625 - actor/entropy:0.34878426790237427 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35282185673713684 - training/rollout_probs_diff_mean:0.003718312829732895 - training/rollout_probs_diff_std:0.009631498716771603 - training/rollout_actor_probs_pearson_corr:0.999345600605011 - rollout_corr/rollout_is_mean:1.000012993812561 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.0160612621111795e-05 - rollout_corr/rollout_is_max:1.975199818611145 - rollout_corr/rollout_is_min:0.19730398058891296 - rollout_corr/rollout_is_std:0.03194605931639671 - rollout_corr/rollout_is_eff_sample_size:0.9989806087004571 - rollout_corr/rollout_is_seq_mean:1.0000489950180054 - rollout_corr/rollout_is_seq_std:0.0017589139752089977 - rollout_corr/rollout_is_seq_max:1.0087381601333618 - rollout_corr/rollout_is_seq_min:0.992647647857666 - rollout_corr/rollout_is_seq_max_deviation:0.008738160133361816 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3483932986855507) - rollout_corr/training_log_ppl:np.float64(0.2762820063435356) - rollout_corr/kl:np.float64(0.0006076907076355553) - rollout_corr/k3_kl:np.float64(0.0005228528713416836) - rollout_corr/rollout_ppl:np.float64(1.347519775852561) - rollout_corr/rollout_log_ppl:np.float64(0.27567431312490953) - rollout_corr/log_ppl_diff:np.float64(0.0006076932186260819) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012910107034258544) - rollout_corr/log_ppl_diff_max:np.float64(0.009554415941238403) - rollout_corr/log_ppl_diff_min:np.float64(-0.007596790790557861) - rollout_corr/ppl_ratio:np.float64(1.0006093357224017) - rollout_corr/chi2_token:np.float64(0.0008673274423927069) - rollout_corr/chi2_seq:np.float64(0.4545943159610033) - actor/pg_loss:np.float64(4.701910074800253e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021670004468887782) - actor/ppo_kl:np.float64(0.0001372517156355002) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2546451985836029) - perf/mfu/actor:np.float64(0.07849234372480743) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.89488983154297) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/reward/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.09773915943586196) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8506) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.07051862491369472) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.6758124999999999) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.06817587534083876) - val-aux/sciknoweval/score/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/score/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/score/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.09773915943586196) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8506) - val-aux/sciknoweval/score/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.07051862491369472) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.6758124999999999) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.06817587534083876) - val-core/sciknoweval/acc/mean@16:np.float64(0.63828125) - val-aux/sciknoweval/acc/std@16:np.float64(0.18977686952476835) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7111625) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15430243477041897) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.56455) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.1662157425344542) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6377124999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.19070491787466565) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7680125) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11699107745399087) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5010874999999999) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1336089734386206) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6575625) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1445571378929248) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8131625) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.08665177217150197) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.44706250000000003) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.10195007783293546) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6680875000000001) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.09773915943586196) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8506) - val-core/sciknoweval/acc/best@16/std:np.float64(0.061461459372971074) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40423749999999997) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.07051862491369472) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.6758124999999999) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.06817587534083876) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.98125) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.041294709008662854) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9961125) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.016882665811088163) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.966925) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.04991881605672479) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9813124999999999) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.04115652748695172) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9996875) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.004110435910939457) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9449500000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.05482807368910221) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9914250000000001) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.027274560942092924) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.92125) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.05037233272292474) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9971) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.01160302730045884) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8982125) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.037001893862708676) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9991249999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.005012406355781736) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:70 - training/epoch:3 - critic/score/mean:0.65234375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.65234375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0005754174781031907 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0005754174781031907 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:581.26953125 - response_length/max:2622.0 - response_length/min:98.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:581.26953125 - response_length_non_aborted/max:2622.0 - response_length_non_aborted/min:98.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:267.65625 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7524307370185852e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.118723189458251) - timing_s/agent_loop/generate_sequences/max:np.float64(16.3854166790843) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.583953251349158) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.3854166790843) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:2622 - timing_s/gen:22.14735895767808 - timing_s/reward:0.0681693684309721 - timing_s/old_log_prob:2.4311879333108664 - timing_s/adv:0.11663812026381493 - timing_s/update_actor:9.943649336695671 - timing_s/step:34.796333791688085 - timing_s/testing:115.27732796221972 - timing_s/save_checkpoint:6.6401550360023975 - timing_s/stop_profile:0.000131901353597641 - timing_per_token_ms/update_actor:0.04575474214515436 - timing_per_token_ms/gen:0.1488347767728106 - timing_per_token_ms/adv:0.0005366990464227075 - perf/total_num_tokens:217325 - perf/time_per_step:34.796333791688085 - perf/throughput:780.7036558112666 (TaskRunner pid=2049544) Training Progress: 70%|███████ | 70/100 [54:53<36:48, 73.62s/it] (TaskRunner pid=2049544) step:71 - global_seqlen/min:20204 - global_seqlen/max:38376 - global_seqlen/minmax_diff:18172 - global_seqlen/balanced_min:28759 - global_seqlen/balanced_max:29243 - global_seqlen/mean:28856.5 - actor/entropy:0.32371971011161804 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.35350361466407776 - training/rollout_probs_diff_mean:0.003485669381916523 - training/rollout_probs_diff_std:0.009408628568053246 - training/rollout_actor_probs_pearson_corr:0.9993824362754822 - rollout_corr/rollout_is_mean:1.0000401735305786 - rollout_corr/rollout_is_ratio_fraction_high:6.257665518205613e-06 - rollout_corr/rollout_is_ratio_fraction_low:6.257665518205613e-06 - rollout_corr/rollout_is_max:3.121652126312256 - rollout_corr/rollout_is_min:0.4736731946468353 - rollout_corr/rollout_is_std:0.03072100132703781 - rollout_corr/rollout_is_eff_sample_size:0.9990571099353601 - rollout_corr/rollout_is_seq_mean:1.0000032186508179 - rollout_corr/rollout_is_seq_std:0.0015029723290354013 - rollout_corr/rollout_is_seq_max:1.0061728954315186 - rollout_corr/rollout_is_seq_min:0.9937267303466797 - rollout_corr/rollout_is_seq_max_deviation:0.0062732696533203125 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3052765680477023) - rollout_corr/training_log_ppl:np.float64(0.24597317739971913) - rollout_corr/kl:np.float64(0.00043000688371730433) - rollout_corr/k3_kl:np.float64(0.000540082649621354) - rollout_corr/rollout_ppl:np.float64(1.3047043732367456) - rollout_corr/rollout_log_ppl:np.float64(0.2455431708949618) - rollout_corr/log_ppl_diff:np.float64(0.0004300065047573298) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012972943659406155) - rollout_corr/log_ppl_diff_max:np.float64(0.00797504186630249) - rollout_corr/log_ppl_diff_min:np.float64(-0.0062088742852211) - rollout_corr/ppl_ratio:np.float64(1.0004315802361816) - rollout_corr/chi2_token:np.float64(0.0013474705629050732) - rollout_corr/chi2_seq:np.float64(1.4930782679002732) - actor/pg_loss:np.float64(0.0001349709928035736) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00038389957057916035) - actor/ppo_kl:np.float64(-3.772802561208266e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.17576251924037933) - perf/mfu/actor:np.float64(0.08244675822096596) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.2421646118164) - actor/lr:np.float64(1e-06) - training/global_step:71 - training/epoch:3 - critic/score/mean:0.75 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.009262127801775932 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.009262127801775932 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:624.234375 - response_length/max:3404.0 - response_length/min:107.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:624.234375 - response_length_non_aborted/max:3404.0 - response_length_non_aborted/min:107.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.53125 - prompt_length/max:437.0 - prompt_length/min:170.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012946873903274536 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2765609622001648) - timing_s/agent_loop/generate_sequences/max:np.float64(20.10214041545987) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.958212114004709) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.10214041545987) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:3404 - timing_s/gen:25.802896425127983 - timing_s/reward:0.07092655450105667 - timing_s/old_log_prob:2.582106838002801 - timing_s/adv:0.1316581778228283 - timing_s/update_actor:10.102314312011003 - timing_s/step:38.781780099496245 - timing_s/stop_profile:0.00012562982738018036 - timing_per_token_ms/update_actor:0.04376099974014089 - timing_per_token_ms/gen:0.16146589838256853 - timing_per_token_ms/adv:0.0005703142178661146 - perf/total_num_tokens:230852 - perf/time_per_step:38.781780099496245 - perf/throughput:744.0736326689355 (TaskRunner pid=2049544) Training Progress: 71%|███████ | 71/100 [55:32<30:32, 63.19s/it] (TaskRunner pid=2049544) step:72 - global_seqlen/min:24019 - global_seqlen/max:35372 - global_seqlen/minmax_diff:11353 - global_seqlen/balanced_min:30681 - global_seqlen/balanced_max:31067 - global_seqlen/mean:30756.75 - actor/entropy:0.2652339041233063 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5334677696228027 - training/rollout_probs_diff_mean:0.003048401791602373 - training/rollout_probs_diff_std:0.009212449193000793 - training/rollout_actor_probs_pearson_corr:0.9992727637290955 - rollout_corr/rollout_is_mean:1.0000463724136353 - rollout_corr/rollout_is_ratio_fraction_high:1.1159842870256398e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.6739762941142544e-05 - rollout_corr/rollout_is_max:2.703545570373535 - rollout_corr/rollout_is_min:0.2786954641342163 - rollout_corr/rollout_is_std:0.02862177975475788 - rollout_corr/rollout_is_eff_sample_size:0.9991814643117487 - rollout_corr/rollout_is_seq_mean:1.0000512599945068 - rollout_corr/rollout_is_seq_std:0.0012803305871784687 - rollout_corr/rollout_is_seq_max:1.005553960800171 - rollout_corr/rollout_is_seq_min:0.9965124726295471 - rollout_corr/rollout_is_seq_max_deviation:0.0055539608001708984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.272761202417314) - rollout_corr/training_log_ppl:np.float64(0.2287760855979286) - rollout_corr/kl:np.float64(0.0003352667156821809) - rollout_corr/k3_kl:np.float64(0.0004365081685619998) - rollout_corr/rollout_ppl:np.float64(1.2723114336840808) - rollout_corr/rollout_log_ppl:np.float64(0.22844081881339662) - rollout_corr/log_ppl_diff:np.float64(0.00033526678453199565) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010565653501544148) - rollout_corr/log_ppl_diff_max:np.float64(0.00969955325126648) - rollout_corr/log_ppl_diff_min:np.float64(-0.005003869533538818) - rollout_corr/ppl_ratio:np.float64(1.0003363261930645) - rollout_corr/chi2_token:np.float64(0.0010922125075012445) - rollout_corr/chi2_seq:np.float64(1.312244726344943) - actor/pg_loss:np.float64(8.467817679047585e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000129777238043971) - actor/ppo_kl:np.float64(-3.5065210759910315e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1640554666519165) - perf/mfu/actor:np.float64(0.08815191325739583) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.06222915649414) - actor/lr:np.float64(1e-06) - training/global_step:72 - training/epoch:3 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006141400896012783 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006141400896012783 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:700.0546875 - response_length/max:3313.0 - response_length/min:189.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:700.0546875 - response_length_non_aborted/max:3313.0 - response_length_non_aborted/min:189.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.09375 - prompt_length/max:355.0 - prompt_length/min:178.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013216771185398102 - timing_s/agent_loop/generate_sequences/min:np.float64(2.07568240724504) - timing_s/agent_loop/generate_sequences/max:np.float64(20.67868852801621) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.946097736770753) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.67868852801621) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:246 - timing_s/agent_loop/slowest/response_length:3313 - timing_s/gen:26.445967487990856 - timing_s/reward:0.0720598753541708 - timing_s/old_log_prob:2.519180826842785 - timing_s/adv:0.12123813852667809 - timing_s/update_actor:10.210091643035412 - timing_s/step:39.45813396573067 - timing_s/stop_profile:0.00012371689081192017 - timing_per_token_ms/update_actor:0.04149532884259314 - timing_per_token_ms/gen:0.14756641494520994 - timing_per_token_ms/adv:0.0004927298012902781 - perf/total_num_tokens:246054 - perf/time_per_step:39.45813396573067 - perf/throughput:779.4780672272083 (TaskRunner pid=2049544) Training Progress: 72%|███████▏ | 72/100 [56:11<26:10, 56.08s/it] (TaskRunner pid=2049544) step:73 - global_seqlen/min:21925 - global_seqlen/max:46347 - global_seqlen/minmax_diff:24422 - global_seqlen/balanced_min:36563 - global_seqlen/balanced_max:36628 - global_seqlen/mean:36610.375 - actor/entropy:0.3390262722969055 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.43396425247192383 - training/rollout_probs_diff_mean:0.0032661668956279755 - training/rollout_probs_diff_std:0.008885001763701439 - training/rollout_actor_probs_pearson_corr:0.9994308948516846 - rollout_corr/rollout_is_mean:0.9999864101409912 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.7025687813875265e-05 - rollout_corr/rollout_is_max:1.9699597358703613 - rollout_corr/rollout_is_min:0.2820616662502289 - rollout_corr/rollout_is_std:0.029777150601148605 - rollout_corr/rollout_is_eff_sample_size:0.9991141068069422 - rollout_corr/rollout_is_seq_mean:0.9999426007270813 - rollout_corr/rollout_is_seq_std:0.0015595207223668694 - rollout_corr/rollout_is_seq_max:1.0064419507980347 - rollout_corr/rollout_is_seq_min:0.992620587348938 - rollout_corr/rollout_is_seq_max_deviation:0.007379412651062012 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4027120932005346) - rollout_corr/training_log_ppl:np.float64(0.31371801071509253) - rollout_corr/kl:np.float64(0.000663529713321509) - rollout_corr/k3_kl:np.float64(0.0005334890627182176) - rollout_corr/rollout_ppl:np.float64(1.4016525098122656) - rollout_corr/rollout_log_ppl:np.float64(0.3130544779851334) - rollout_corr/log_ppl_diff:np.float64(0.0006635327299591154) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013385674392338842) - rollout_corr/log_ppl_diff_max:np.float64(0.008049964904785156) - rollout_corr/log_ppl_diff_min:np.float64(-0.005723848938941956) - rollout_corr/ppl_ratio:np.float64(1.0006653110031039) - rollout_corr/chi2_token:np.float64(0.0007862187922000885) - rollout_corr/chi2_seq:np.float64(0.7408852751832455) - actor/pg_loss:np.float64(-9.118067100644112e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00025507958866910485) - actor/ppo_kl:np.float64(9.864521248204028e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.22031916305422783) - perf/mfu/actor:np.float64(0.10239718785831436) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.18194198608398) - actor/lr:np.float64(1e-06) - training/global_step:73 - training/epoch:3 - critic/score/mean:0.61328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.61328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006957425735890865 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006957425735890865 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:867.23046875 - response_length/max:3153.0 - response_length/min:79.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:867.23046875 - response_length_non_aborted/max:3153.0 - response_length_non_aborted/min:79.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.84375 - prompt_length/max:375.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010944902896881104 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9543074388056993) - timing_s/agent_loop/generate_sequences/max:np.float64(21.861840976402164) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.284751043880533) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.861840976402164) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:309 - timing_s/agent_loop/slowest/response_length:3153 - timing_s/gen:27.61041198298335 - timing_s/reward:0.07980250380933285 - timing_s/old_log_prob:2.6316978093236685 - timing_s/adv:0.14758859016001225 - timing_s/update_actor:10.437388250604272 - timing_s/step:40.99476144276559 - timing_s/stop_profile:0.00011257268488407135 - timing_per_token_ms/update_actor:0.03563671585788274 - timing_per_token_ms/gen:0.12436506291572648 - timing_per_token_ms/adv:0.00050391654742683 - perf/total_num_tokens:292883 - perf/time_per_step:40.99476144276559 - perf/throughput:893.0500803404648 (TaskRunner pid=2049544) Training Progress: 73%|███████▎ | 73/100 [56:52<23:12, 51.57s/it] (TaskRunner pid=2049544) step:74 - global_seqlen/min:18401 - global_seqlen/max:45796 - global_seqlen/minmax_diff:27395 - global_seqlen/balanced_min:30702 - global_seqlen/balanced_max:30981 - global_seqlen/mean:30746.0 - actor/entropy:0.31715160608291626 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4399150013923645 - training/rollout_probs_diff_mean:0.0034521878696978092 - training/rollout_probs_diff_std:0.009627916850149632 - training/rollout_actor_probs_pearson_corr:0.9993271231651306 - rollout_corr/rollout_is_mean:1.0001097917556763 - rollout_corr/rollout_is_ratio_fraction_high:1.1016612916137092e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.652492028370034e-05 - rollout_corr/rollout_is_max:2.2787067890167236 - rollout_corr/rollout_is_min:0.41993895173072815 - rollout_corr/rollout_is_std:0.030713239684700966 - rollout_corr/rollout_is_eff_sample_size:0.9990577048586795 - rollout_corr/rollout_is_seq_mean:1.000206708908081 - rollout_corr/rollout_is_seq_std:0.002247885102406144 - rollout_corr/rollout_is_seq_max:1.0164759159088135 - rollout_corr/rollout_is_seq_min:0.9938144683837891 - rollout_corr/rollout_is_seq_max_deviation:0.016475915908813477 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3208666159771383) - rollout_corr/training_log_ppl:np.float64(0.2606007744325325) - rollout_corr/kl:np.float64(0.0003702222969650393) - rollout_corr/k3_kl:np.float64(0.000720645587279023) - rollout_corr/rollout_ppl:np.float64(1.3202885040082037) - rollout_corr/rollout_log_ppl:np.float64(0.26023055279802065) - rollout_corr/log_ppl_diff:np.float64(0.0003702216345118359) - rollout_corr/log_ppl_abs_diff:np.float64(0.0016211537149501964) - rollout_corr/log_ppl_diff_max:np.float64(0.01012013852596283) - rollout_corr/log_ppl_diff_min:np.float64(-0.012066826224327087) - rollout_corr/ppl_ratio:np.float64(1.0003729804884642) - rollout_corr/chi2_token:np.float64(0.0021484598983079195) - rollout_corr/chi2_seq:np.float64(2.6817988054826856) - actor/pg_loss:np.float64(-0.00016311672516167164) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00024571236758674786) - actor/ppo_kl:np.float64(-5.575577448624358e-06) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21021737158298492) - perf/mfu/actor:np.float64(0.08564326289574913) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.05369186401367) - actor/lr:np.float64(1e-06) - training/global_step:74 - training/epoch:3 - critic/score/mean:0.75390625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.75390625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008819486945867538 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008819486945867538 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:709.15625 - response_length/max:4610.0 - response_length/min:62.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:709.15625 - response_length_non_aborted/max:4610.0 - response_length_non_aborted/min:62.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:251.65625 - prompt_length/max:353.0 - prompt_length/min:143.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.643472731113434e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.46839356422424316) - timing_s/agent_loop/generate_sequences/max:np.float64(26.743163952603936) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.783247412800847) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.743163952603936) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:297 - timing_s/agent_loop/slowest/response_length:4610 - timing_s/gen:33.11821131967008 - timing_s/reward:0.07101857103407383 - timing_s/old_log_prob:2.5942629985511303 - timing_s/adv:0.12228076718747616 - timing_s/update_actor:10.489771209657192 - timing_s/step:46.487874053418636 - timing_s/stop_profile:0.00010906346142292023 - timing_per_token_ms/update_actor:0.042646893944160186 - timing_per_token_ms/gen:0.18242525954958622 - timing_per_token_ms/adv:0.0004971409581225044 - perf/total_num_tokens:245968 - perf/time_per_step:46.487874053418636 - perf/throughput:661.3767703093963 (TaskRunner pid=2049544) Training Progress: 74%|███████▍ | 74/100 [57:39<21:41, 50.06s/it] (TaskRunner pid=2049544) step:75 - global_seqlen/min:19402 - global_seqlen/max:28818 - global_seqlen/minmax_diff:9416 - global_seqlen/balanced_min:23785 - global_seqlen/balanced_max:23996 - global_seqlen/mean:23814.375 - actor/entropy:0.25116294622421265 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135130643844604 - training/rollout_probs_diff_mean:0.0031402192544192076 - training/rollout_probs_diff_std:0.009609016589820385 - training/rollout_actor_probs_pearson_corr:0.9991532564163208 - rollout_corr/rollout_is_mean:1.0000470876693726 - rollout_corr/rollout_is_ratio_fraction_high:8.058602361415979e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4175806174753234e-05 - rollout_corr/rollout_is_max:2.1670117378234863 - rollout_corr/rollout_is_min:0.4087337255477905 - rollout_corr/rollout_is_std:0.03037172183394432 - rollout_corr/rollout_is_eff_sample_size:0.9990784086315643 - rollout_corr/rollout_is_seq_mean:1.0000663995742798 - rollout_corr/rollout_is_seq_std:0.0019097818294540048 - rollout_corr/rollout_is_seq_max:1.0100950002670288 - rollout_corr/rollout_is_seq_min:0.9921054244041443 - rollout_corr/rollout_is_seq_max_deviation:0.010095000267028809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2536835842765868) - rollout_corr/training_log_ppl:np.float64(0.20439182232803432) - rollout_corr/kl:np.float64(0.00044191136634275097) - rollout_corr/k3_kl:np.float64(0.0004956063722261206) - rollout_corr/rollout_ppl:np.float64(1.2531235865317285) - rollout_corr/rollout_log_ppl:np.float64(0.20394990891509224) - rollout_corr/log_ppl_diff:np.float64(0.0004419134129420854) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013838859085808508) - rollout_corr/log_ppl_diff_max:np.float64(0.009681947529315948) - rollout_corr/log_ppl_diff_min:np.float64(-0.005931958556175232) - rollout_corr/ppl_ratio:np.float64(1.0004437156021595) - rollout_corr/chi2_token:np.float64(0.0011102964635938406) - rollout_corr/chi2_seq:np.float64(0.39504921343177557) - actor/pg_loss:np.float64(1.7428305000066757e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018432266847412393) - actor/ppo_kl:np.float64(3.7105337884923983e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16733924113214016) - perf/mfu/actor:np.float64(0.06918613903772251) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(106.97526931762695) - actor/lr:np.float64(1e-06) - training/global_step:75 - training/epoch:3 - critic/score/mean:0.859375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.859375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0008511898340657353 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0008511898340657353 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:484.73046875 - response_length/max:2220.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:484.73046875 - response_length_non_aborted/max:2220.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.46875 - prompt_length/max:383.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00016090832650661469 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2748903073370457) - timing_s/agent_loop/generate_sequences/max:np.float64(13.663885425776243) - timing_s/agent_loop/generate_sequences/mean:np.float64(4.775116956734564) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(13.663885425776243) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2220 - timing_s/gen:19.10645161010325 - timing_s/reward:0.06184021010994911 - timing_s/old_log_prob:2.374053070321679 - timing_s/adv:0.11829666048288345 - timing_s/update_actor:9.632479302585125 - timing_s/step:31.360355062410235 - timing_s/stop_profile:3.0426308512687683e-05 - timing_per_token_ms/update_actor:0.05056021469482783 - timing_per_token_ms/gen:0.15397129211710156 - timing_per_token_ms/adv:0.0006209309528534942 - perf/total_num_tokens:190515 - perf/time_per_step:31.360355062410235 - perf/throughput:759.3783601176395 (TaskRunner pid=2049544) Training Progress: 75%|███████▌ | 75/100 [58:10<18:31, 44.46s/it] (TaskRunner pid=2049544) step:76 - global_seqlen/min:25932 - global_seqlen/max:42011 - global_seqlen/minmax_diff:16079 - global_seqlen/balanced_min:31987 - global_seqlen/balanced_max:32015 - global_seqlen/mean:32005.0 - actor/entropy:0.35323843359947205 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8143891096115112 - training/rollout_probs_diff_mean:0.003405920695513487 - training/rollout_probs_diff_std:0.009273294359445572 - training/rollout_actor_probs_pearson_corr:0.9994474053382874 - rollout_corr/rollout_is_mean:1.0001150369644165 - rollout_corr/rollout_is_ratio_fraction_high:1.081595564755844e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.081595564755844e-05 - rollout_corr/rollout_is_max:3.5232980251312256 - rollout_corr/rollout_is_min:0.16788354516029358 - rollout_corr/rollout_is_std:0.030408985912799835 - rollout_corr/rollout_is_eff_sample_size:0.9990763858113328 - rollout_corr/rollout_is_seq_mean:1.0001401901245117 - rollout_corr/rollout_is_seq_std:0.0016239474061876535 - rollout_corr/rollout_is_seq_max:1.010636329650879 - rollout_corr/rollout_is_seq_min:0.9931896328926086 - rollout_corr/rollout_is_seq_max_deviation:0.010636329650878906 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3853302095085382) - rollout_corr/training_log_ppl:np.float64(0.3024687277356861) - rollout_corr/kl:np.float64(0.0005689695157187202) - rollout_corr/k3_kl:np.float64(0.0005752426737899441) - rollout_corr/rollout_ppl:np.float64(1.38448647223413) - rollout_corr/rollout_log_ppl:np.float64(0.30189975767279975) - rollout_corr/log_ppl_diff:np.float64(0.0005689700628863648) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012543359625851735) - rollout_corr/log_ppl_diff_max:np.float64(0.007834061980247498) - rollout_corr/log_ppl_diff_min:np.float64(-0.006428062915802002) - rollout_corr/ppl_ratio:np.float64(1.0005704772192985) - rollout_corr/chi2_token:np.float64(0.0012044101022183895) - rollout_corr/chi2_seq:np.float64(1.3872064049355686) - actor/pg_loss:np.float64(5.75545709580183e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002793574865336268) - actor/ppo_kl:np.float64(0.00019198946114684645) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3208663538098335) - perf/mfu/actor:np.float64(0.09134199443488283) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.03634643554688) - actor/lr:np.float64(1e-06) - training/global_step:76 - training/epoch:3 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008177538402378559 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008177538402378559 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:722.3125 - response_length/max:2396.0 - response_length/min:87.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:722.3125 - response_length_non_aborted/max:2396.0 - response_length_non_aborted/min:87.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.84375 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:3.119371831417084e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.99646220728755) - timing_s/agent_loop/generate_sequences/max:np.float64(16.732603766024113) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.14587117084011) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(16.732603766024113) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:306 - timing_s/agent_loop/slowest/response_length:2371 - timing_s/gen:22.66927962191403 - timing_s/reward:0.07421145401895046 - timing_s/old_log_prob:2.5292600374668837 - timing_s/adv:0.1157302763313055 - timing_s/update_actor:10.305034121498466 - timing_s/step:35.780901934951544 - timing_s/stop_profile:0.00012325309216976166 - timing_per_token_ms/update_actor:0.040247750826036814 - timing_per_token_ms/gen:0.12259496204634653 - timing_per_token_ms/adv:0.00045200076679934975 - perf/total_num_tokens:256040 - perf/time_per_step:35.780901934951544 - perf/throughput:894.471583141867 (TaskRunner pid=2049544) Training Progress: 76%|███████▌ | 76/100 [58:46<16:44, 41.87s/it] (TaskRunner pid=2049544) step:77 - global_seqlen/min:16244 - global_seqlen/max:41465 - global_seqlen/minmax_diff:25221 - global_seqlen/balanced_min:29420 - global_seqlen/balanced_max:29517 - global_seqlen/mean:29492.5 - actor/entropy:0.3045775294303894 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2915552258491516 - training/rollout_probs_diff_mean:0.0032992891501635313 - training/rollout_probs_diff_std:0.009267176501452923 - training/rollout_actor_probs_pearson_corr:0.9993472695350647 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:5.8924742916133255e-06 - rollout_corr/rollout_is_ratio_fraction_low:0.0 - rollout_corr/rollout_is_max:3.140592098236084 - rollout_corr/rollout_is_min:0.5226767659187317 - rollout_corr/rollout_is_std:0.029582349583506584 - rollout_corr/rollout_is_eff_sample_size:0.9991255307618554 - rollout_corr/rollout_is_seq_mean:1.0000417232513428 - rollout_corr/rollout_is_seq_std:0.0017432396998628974 - rollout_corr/rollout_is_seq_max:1.008344054222107 - rollout_corr/rollout_is_seq_min:0.9936603307723999 - rollout_corr/rollout_is_seq_max_deviation:0.008344054222106934 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2967352382838726) - rollout_corr/training_log_ppl:np.float64(0.24739519167633262) - rollout_corr/kl:np.float64(0.0005334063558146696) - rollout_corr/k3_kl:np.float64(0.0005630067024355867) - rollout_corr/rollout_ppl:np.float64(1.2960236202925444) - rollout_corr/rollout_log_ppl:np.float64(0.24686178589763585) - rollout_corr/log_ppl_diff:np.float64(0.0005334057786967605) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014061912952456623) - rollout_corr/log_ppl_diff_max:np.float64(0.01065683364868164) - rollout_corr/log_ppl_diff_min:np.float64(-0.0076304227113723755) - rollout_corr/ppl_ratio:np.float64(1.0005355728790164) - rollout_corr/chi2_token:np.float64(0.0011709912214428186) - rollout_corr/chi2_seq:np.float64(0.44916587066836655) - actor/pg_loss:np.float64(2.6725465431809425e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00018346831632243266) - actor/ppo_kl:np.float64(9.181251817746983e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16028263419866562) - perf/mfu/actor:np.float64(0.08482457572663438) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.09542083740234) - actor/lr:np.float64(1e-06) - training/global_step:77 - training/epoch:3 - critic/score/mean:0.703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006819802336394787 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006819802336394787 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:662.921875 - response_length/max:2972.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:662.921875 - response_length_non_aborted/max:2972.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:258.71875 - prompt_length/max:355.0 - prompt_length/min:197.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00013144128024578094 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8779960200190544) - timing_s/agent_loop/generate_sequences/max:np.float64(18.480581056326628) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.09187524789013) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.480581056326628) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:315 - timing_s/agent_loop/slowest/response_length:2972 - timing_s/gen:24.529541220515966 - timing_s/reward:0.07079425640404224 - timing_s/old_log_prob:2.487271063029766 - timing_s/adv:0.11753782816231251 - timing_s/update_actor:10.21386044844985 - timing_s/step:37.50457299686968 - timing_s/stop_profile:2.83215194940567e-05 - timing_per_token_ms/update_actor:0.043290075648257396 - timing_per_token_ms/gen:0.1445396871126639 - timing_per_token_ms/adv:0.0004981682977126071 - perf/total_num_tokens:235940 - perf/time_per_step:37.50457299686968 - perf/throughput:786.3707714379681 (TaskRunner pid=2049544) Training Progress: 77%|███████▋ | 77/100 [59:24<15:33, 40.57s/it] (TaskRunner pid=2049544) step:78 - global_seqlen/min:16539 - global_seqlen/max:34942 - global_seqlen/minmax_diff:18403 - global_seqlen/balanced_min:26643 - global_seqlen/balanced_max:26991 - global_seqlen/mean:26772.625 - actor/entropy:0.324239581823349 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.30065640807151794 - training/rollout_probs_diff_mean:0.003476528450846672 - training/rollout_probs_diff_std:0.009609517641365528 - training/rollout_actor_probs_pearson_corr:0.9993136525154114 - rollout_corr/rollout_is_mean:0.9999270439147949 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.365252956020413e-05 - rollout_corr/rollout_is_max:1.927759051322937 - rollout_corr/rollout_is_min:0.17178574204444885 - rollout_corr/rollout_is_std:0.031049082055687904 - rollout_corr/rollout_is_eff_sample_size:0.9990367639843101 - rollout_corr/rollout_is_seq_mean:0.9997465014457703 - rollout_corr/rollout_is_seq_std:0.0017120024422183633 - rollout_corr/rollout_is_seq_max:1.0067331790924072 - rollout_corr/rollout_is_seq_min:0.9950670599937439 - rollout_corr/rollout_is_seq_max_deviation:0.0067331790924072266 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3026048908941448) - rollout_corr/training_log_ppl:np.float64(0.24134253649390303) - rollout_corr/kl:np.float64(0.0007427987794557112) - rollout_corr/k3_kl:np.float64(0.0005388006466660045) - rollout_corr/rollout_ppl:np.float64(1.301613979972899) - rollout_corr/rollout_log_ppl:np.float64(0.24059973818657454) - rollout_corr/log_ppl_diff:np.float64(0.000742798307328485) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014510368491755798) - rollout_corr/log_ppl_diff_max:np.float64(0.0068367719650268555) - rollout_corr/log_ppl_diff_min:np.float64(-0.005617767572402954) - rollout_corr/ppl_ratio:np.float64(1.0007447020616382) - rollout_corr/chi2_token:np.float64(0.0006613847799599171) - rollout_corr/chi2_seq:np.float64(0.6144026103429496) - actor/pg_loss:np.float64(5.164649337530136e-06) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00011791538452143868) - actor/ppo_kl:np.float64(-3.060101226282086e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16195594891905785) - perf/mfu/actor:np.float64(0.07693655141257658) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.07821655273438) - actor/lr:np.float64(1e-06) - training/global_step:78 - training/epoch:3 - critic/score/mean:0.78125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.78125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0012876042164862156 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0012876042164862156 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:572.23828125 - response_length/max:3226.0 - response_length/min:104.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:572.23828125 - response_length_non_aborted/max:3226.0 - response_length_non_aborted/min:104.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.40625 - prompt_length/max:355.0 - prompt_length/min:181.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.397596836090088e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2444690354168415) - timing_s/agent_loop/generate_sequences/max:np.float64(19.064789429306984) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.27938631172583) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(19.064789429306984) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:251 - timing_s/agent_loop/slowest/response_length:3226 - timing_s/gen:24.83233668655157 - timing_s/reward:0.06715011782944202 - timing_s/old_log_prob:2.454452695325017 - timing_s/adv:0.12591312639415264 - timing_s/update_actor:10.206827508285642 - timing_s/step:37.77338271215558 - timing_s/stop_profile:0.00012371130287647247 - timing_per_token_ms/update_actor:0.04765514918823631 - timing_per_token_ms/gen:0.1695121042408277 - timing_per_token_ms/adv:0.0005878818681122632 - perf/total_num_tokens:214181 - perf/time_per_step:37.77338271215558 - perf/throughput:708.7695905875142 (TaskRunner pid=2049544) Training Progress: 78%|███████▊ | 78/100 [1:00:02<14:34, 39.74s/it] (TaskRunner pid=2049544) step:79 - global_seqlen/min:20249 - global_seqlen/max:46165 - global_seqlen/minmax_diff:25916 - global_seqlen/balanced_min:30262 - global_seqlen/balanced_max:30358 - global_seqlen/mean:30336.375 - actor/entropy:0.27736997604370117 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4035380482673645 - training/rollout_probs_diff_mean:0.0030377230141311884 - training/rollout_probs_diff_std:0.009076771326363087 - training/rollout_actor_probs_pearson_corr:0.9993468523025513 - rollout_corr/rollout_is_mean:0.9998963475227356 - rollout_corr/rollout_is_ratio_fraction_high:5.790153409179766e-06 - rollout_corr/rollout_is_ratio_fraction_low:5.211137977312319e-05 - rollout_corr/rollout_is_max:4.029329299926758 - rollout_corr/rollout_is_min:0.07581690698862076 - rollout_corr/rollout_is_std:0.02898186817765236 - rollout_corr/rollout_is_eff_sample_size:0.9991605182489611 - rollout_corr/rollout_is_seq_mean:0.9999415874481201 - rollout_corr/rollout_is_seq_std:0.0017486511496827006 - rollout_corr/rollout_is_seq_max:1.0085891485214233 - rollout_corr/rollout_is_seq_min:0.9941185712814331 - rollout_corr/rollout_is_seq_max_deviation:0.00858914852142334 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2839201726019382) - rollout_corr/training_log_ppl:np.float64(0.2279481466684956) - rollout_corr/kl:np.float64(0.0005260294094213691) - rollout_corr/k3_kl:np.float64(0.0005059725925065095) - rollout_corr/rollout_ppl:np.float64(1.2831925540231168) - rollout_corr/rollout_log_ppl:np.float64(0.2274221143743489) - rollout_corr/log_ppl_diff:np.float64(0.0005260322941467166) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012799371324945241) - rollout_corr/log_ppl_diff_max:np.float64(0.008957505226135254) - rollout_corr/log_ppl_diff_min:np.float64(-0.0052579790353775024) - rollout_corr/ppl_ratio:np.float64(1.0005276040174067) - rollout_corr/chi2_token:np.float64(0.0009765762370079756) - rollout_corr/chi2_seq:np.float64(0.7638932466506958) - actor/pg_loss:np.float64(0.00014414661563932896) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00017305412563928257) - actor/ppo_kl:np.float64(-2.993323226707645e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.20632774755358696) - perf/mfu/actor:np.float64(0.08763345813710535) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.01940155029297) - actor/lr:np.float64(1e-06) - training/global_step:79 - training/epoch:3 - critic/score/mean:0.8046875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.8046875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.008221293799579144 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.008221293799579144 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:674.63671875 - response_length/max:2725.0 - response_length/min:103.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:674.63671875 - response_length_non_aborted/max:2725.0 - response_length_non_aborted/min:103.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:273.375 - prompt_length/max:365.0 - prompt_length/min:171.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010620616376399994 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1851273197680712) - timing_s/agent_loop/generate_sequences/max:np.float64(17.865634333342314) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.507378077250905) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.865634333342314) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:308 - timing_s/agent_loop/slowest/response_length:2725 - timing_s/gen:23.788950365036726 - timing_s/reward:0.07290799915790558 - timing_s/old_log_prob:2.4474152121692896 - timing_s/adv:0.1101104523986578 - timing_s/update_actor:10.210453910753131 - timing_s/step:36.71885127387941 - timing_s/stop_profile:0.00012296251952648163 - timing_per_token_ms/update_actor:0.04207182759456729 - timing_per_token_ms/gen:0.13774166863553142 - timing_per_token_ms/adv:0.0004537063689986765 - perf/total_num_tokens:242691 - perf/time_per_step:36.71885127387941 - perf/throughput:826.1798489752948 (TaskRunner pid=2049544) Training Progress: 79%|███████▉ | 79/100 [1:00:38<13:35, 38.85s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 80} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) The electric field $ E $ due to a point charge $ q $ at a distance $ r $ is given by: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) - $ E = 2.0 \, \text{N/C} $ (TaskRunner pid=2049544) - $ r = 50 \, \text{cm} = 0.50 \, \text{m} $ (TaskRunner pid=2049544) - $ k = 8.99 \times 10^9 \, \text{N m}^2/\text{C}^2 $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We need to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.5}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.5}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \text{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \text{C} = 55.6 \, \text{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_70/actor (TaskRunner pid=2049544) step:80 - global_seqlen/min:18167 - global_seqlen/max:36780 - global_seqlen/minmax_diff:18613 - global_seqlen/balanced_min:28127 - global_seqlen/balanced_max:28499 - global_seqlen/mean:28202.25 - actor/entropy:0.23095937073230743 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.44294559955596924 - training/rollout_probs_diff_mean:0.002918842015787959 - training/rollout_probs_diff_std:0.0094361687079072 - training/rollout_actor_probs_pearson_corr:0.9991447329521179 - rollout_corr/rollout_is_mean:0.9999209046363831 - rollout_corr/rollout_is_ratio_fraction_high:1.9175455236108974e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.9175455236108974e-05 - rollout_corr/rollout_is_max:2.320603370666504 - rollout_corr/rollout_is_min:0.33252665400505066 - rollout_corr/rollout_is_std:0.028494464233517647 - rollout_corr/rollout_is_eff_sample_size:0.999188605215701 - rollout_corr/rollout_is_seq_mean:0.9999192953109741 - rollout_corr/rollout_is_seq_std:0.001614381093531847 - rollout_corr/rollout_is_seq_max:1.0071684122085571 - rollout_corr/rollout_is_seq_min:0.9931057691574097 - rollout_corr/rollout_is_seq_max_deviation:0.007168412208557129 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2288768324069679) - rollout_corr/training_log_ppl:np.float64(0.19351409275259357) - rollout_corr/kl:np.float64(0.0005485934744235976) - rollout_corr/k3_kl:np.float64(0.0005014402207166313) - rollout_corr/rollout_ppl:np.float64(1.2281699823215604) - rollout_corr/rollout_log_ppl:np.float64(0.1929654997366015) - rollout_corr/log_ppl_diff:np.float64(0.0005485930159920827) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012800928816432133) - rollout_corr/log_ppl_diff_max:np.float64(0.00843721628189087) - rollout_corr/log_ppl_diff_min:np.float64(-0.006566546857357025) - rollout_corr/ppl_ratio:np.float64(1.0005502616986632) - rollout_corr/chi2_token:np.float64(0.0008651481475681067) - rollout_corr/chi2_seq:np.float64(0.5386446032207459) - actor/pg_loss:np.float64(-2.616771962493658e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001737377756398928) - actor/ppo_kl:np.float64(2.498516215898583e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1959109902381897) - perf/mfu/actor:np.float64(0.08114905418503308) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.12442779541016) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/reward/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10302193213109104) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8416625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04059727090755087) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.683725) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05843863672565752) - val-aux/sciknoweval/score/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/score/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/score/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/score/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/score/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10302193213109104) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8416625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04059727090755087) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.683725) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05843863672565752) - val-core/sciknoweval/acc/mean@16:np.float64(0.6390625) - val-aux/sciknoweval/acc/std@16:np.float64(0.18886344890574386) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7177) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.15087202226244528) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.56135) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.17040711495924588) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6400125) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1892037078209971) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.7728999999999999) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.10375600250036085) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.48866250000000006) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1332606477643524) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6633375) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.15024050549177956) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.812) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.06993051354730821) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4333875) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08524525960422448) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.6773) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10302193213109104) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8416625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.04398837601915231) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.40309999999999996) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04059727090755087) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.683725) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05843863672565752) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.975) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.06354165489638078) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.996175) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.023966893234937735) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9539) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.08044933488834552) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9746749999999998) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.06396939198866787) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998125000000002) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.003145810047315943) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9212250000000001) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.09233866134852486) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9907999999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.03956667018273212) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.8777874999999999) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.08983250151225204) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9978250000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.018387073504573324) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.8370125000000002) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.0672232524014538) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998249999999999) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.004527947017243354) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:80 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0006327900337055326 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0006327900337055326 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:611.1328125 - response_length/max:2845.0 - response_length/min:96.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:611.1328125 - response_length_non_aborted/max:2845.0 - response_length_non_aborted/min:96.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.1875 - prompt_length/max:363.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.000109061598777771 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1131607126444578) - timing_s/agent_loop/generate_sequences/max:np.float64(17.91330724582076) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.142933668168553) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.91330724582076) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:340 - timing_s/agent_loop/slowest/response_length:2845 - timing_s/gen:23.700068602338433 - timing_s/reward:0.06908616237342358 - timing_s/old_log_prob:2.398565139621496 - timing_s/adv:0.14442053809762 - timing_s/update_actor:10.117256434634328 - timing_s/step:36.51560699753463 - timing_s/testing:115.6058545038104 - timing_s/save_checkpoint:6.825648196041584 - timing_s/stop_profile:0.00023227371275424957 - timing_per_token_ms/update_actor:0.04484241698195325 - timing_per_token_ms/gen:0.15148653628851666 - timing_per_token_ms/adv:0.0006401108869754187 - perf/total_num_tokens:225618 - perf/time_per_step:36.51560699753463 - perf/throughput:772.3341419986278 (TaskRunner pid=2049544) Training Progress: 80%|████████ | 80/100 [1:03:17<24:57, 74.90s/it] (TaskRunner pid=2049544) step:81 - global_seqlen/min:22002 - global_seqlen/max:45026 - global_seqlen/minmax_diff:23024 - global_seqlen/balanced_min:31986 - global_seqlen/balanced_max:32078 - global_seqlen/mean:32047.5 - actor/entropy:0.30386513471603394 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24135500192642212 - training/rollout_probs_diff_mean:0.003210825379937887 - training/rollout_probs_diff_std:0.00900829304009676 - training/rollout_actor_probs_pearson_corr:0.9993411898612976 - rollout_corr/rollout_is_mean:1.0000028610229492 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:5.263490493234713e-06 - rollout_corr/rollout_is_max:1.9500842094421387 - rollout_corr/rollout_is_min:0.46151214838027954 - rollout_corr/rollout_is_std:0.029131613671779633 - rollout_corr/rollout_is_eff_sample_size:0.9991520686654056 - rollout_corr/rollout_is_seq_mean:0.9999109506607056 - rollout_corr/rollout_is_seq_std:0.0019725700840353966 - rollout_corr/rollout_is_seq_max:1.0081653594970703 - rollout_corr/rollout_is_seq_min:0.9903681874275208 - rollout_corr/rollout_is_seq_max_deviation:0.009631812572479248 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3244363577105105) - rollout_corr/training_log_ppl:np.float64(0.2665938182544778) - rollout_corr/kl:np.float64(0.0010818528244627146) - rollout_corr/k3_kl:np.float64(0.0008180101519030814) - rollout_corr/rollout_ppl:np.float64(1.3230207040905952) - rollout_corr/rollout_log_ppl:np.float64(0.26551196482614614) - rollout_corr/log_ppl_diff:np.float64(0.0010818534283316694) - rollout_corr/log_ppl_abs_diff:np.float64(0.0018223601437057368) - rollout_corr/log_ppl_diff_max:np.float64(0.01393115520477295) - rollout_corr/log_ppl_diff_min:np.float64(-0.010075211524963379) - rollout_corr/ppl_ratio:np.float64(1.0010864462237805) - rollout_corr/chi2_token:np.float64(0.0015508977230638266) - rollout_corr/chi2_seq:np.float64(1.0036336749326438) - actor/pg_loss:np.float64(7.214734796434641e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002760089199682625) - actor/ppo_kl:np.float64(0.0004848889264348344) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.21540894359350204) - perf/mfu/actor:np.float64(0.0899665554810477) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.27264404296875) - actor/lr:np.float64(1e-06) - training/global_step:81 - training/epoch:3 - critic/score/mean:0.7109375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7109375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.004168026614934206 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.004168026614934206 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:742.140625 - response_length/max:3366.0 - response_length/min:63.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:742.140625 - response_length_non_aborted/max:3366.0 - response_length_non_aborted/min:63.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:259.34375 - prompt_length/max:460.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.953603148460388e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.6883454918861389) - timing_s/agent_loop/generate_sequences/max:np.float64(21.4869614392519) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.608749296545284) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(21.4869614392519) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:211 - timing_s/agent_loop/slowest/response_length:3366 - timing_s/gen:27.376880433410406 - timing_s/reward:0.07466047257184982 - timing_s/old_log_prob:2.649170648306608 - timing_s/adv:0.12651739083230495 - timing_s/update_actor:10.34231029637158 - timing_s/step:40.673489751294255 - timing_s/stop_profile:0.00011551380157470703 - timing_per_token_ms/update_actor:0.04033977024873851 - timing_per_token_ms/gen:0.1440979453092322 - timing_per_token_ms/adv:0.000493476054420411 - perf/total_num_tokens:256380 - perf/time_per_step:40.673489751294255 - perf/throughput:787.921080683278 (TaskRunner pid=2049544) Training Progress: 81%|████████ | 81/100 [1:03:58<20:28, 64.65s/it] (TaskRunner pid=2049544) step:82 - global_seqlen/min:25371 - global_seqlen/max:44114 - global_seqlen/minmax_diff:18743 - global_seqlen/balanced_min:33520 - global_seqlen/balanced_max:33584 - global_seqlen/mean:33563.875 - actor/entropy:0.2920665442943573 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.913862407207489 - training/rollout_probs_diff_mean:0.0031529199331998825 - training/rollout_probs_diff_std:0.009444354102015495 - training/rollout_actor_probs_pearson_corr:0.9993178248405457 - rollout_corr/rollout_is_mean:0.9999819993972778 - rollout_corr/rollout_is_ratio_fraction_high:2.0209470676491037e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.547131175058894e-05 - rollout_corr/rollout_is_max:2.713327169418335 - rollout_corr/rollout_is_min:0.049278318881988525 - rollout_corr/rollout_is_std:0.02968292124569416 - rollout_corr/rollout_is_eff_sample_size:0.9991195807527394 - rollout_corr/rollout_is_seq_mean:0.9999279379844666 - rollout_corr/rollout_is_seq_std:0.0012865856988355517 - rollout_corr/rollout_is_seq_max:1.004623532295227 - rollout_corr/rollout_is_seq_min:0.9942681193351746 - rollout_corr/rollout_is_seq_max_deviation:0.0057318806648254395 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3072700020857155) - rollout_corr/training_log_ppl:np.float64(0.24861305610102136) - rollout_corr/kl:np.float64(0.0007281105056762271) - rollout_corr/k3_kl:np.float64(0.0005667303261134293) - rollout_corr/rollout_ppl:np.float64(1.306269062217325) - rollout_corr/rollout_log_ppl:np.float64(0.2478849448962137) - rollout_corr/log_ppl_diff:np.float64(0.0007281112048076466) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011902832047780976) - rollout_corr/log_ppl_diff_max:np.float64(0.012343257665634155) - rollout_corr/log_ppl_diff_min:np.float64(-0.00298440083861351) - rollout_corr/ppl_ratio:np.float64(1.0007296884432435) - rollout_corr/chi2_token:np.float64(0.0007541878148913383) - rollout_corr/chi2_seq:np.float64(0.7926566984970123) - actor/pg_loss:np.float64(0.0001337927533313632) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021164456336464355) - actor/ppo_kl:np.float64(0.00017838788005519746) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1409779703244567) - perf/mfu/actor:np.float64(0.09575702121449964) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.184974193573) - actor/lr:np.float64(1e-06) - training/global_step:82 - training/epoch:3 - critic/score/mean:0.63671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.63671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.00978454202413559 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.00978454202413559 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:773.15234375 - response_length/max:2703.0 - response_length/min:191.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:773.15234375 - response_length_non_aborted/max:2703.0 - response_length_non_aborted/min:191.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:275.71875 - prompt_length/max:382.0 - prompt_length/min:189.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.613321602344513e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2882147133350372) - timing_s/agent_loop/generate_sequences/max:np.float64(18.499180192127824) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.74611435400584) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.499180192127824) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:240 - timing_s/agent_loop/slowest/response_length:2703 - timing_s/gen:24.018707640469074 - timing_s/reward:0.07196270488202572 - timing_s/old_log_prob:2.514760909602046 - timing_s/adv:0.12197888456285 - timing_s/update_actor:10.373511025682092 - timing_s/step:37.19780015014112 - timing_s/stop_profile:3.86945903301239e-05 - timing_per_token_ms/update_actor:0.03863346762584063 - timing_per_token_ms/gen:0.12135134489215253 - timing_per_token_ms/adv:0.0004542789105952829 - perf/total_num_tokens:268511 - perf/time_per_step:37.19780015014112 - perf/throughput:902.3080629641123 (TaskRunner pid=2049544) Training Progress: 82%|████████▏ | 82/100 [1:04:35<16:55, 56.42s/it] (TaskRunner pid=2049544) step:83 - global_seqlen/min:19276 - global_seqlen/max:41937 - global_seqlen/minmax_diff:22661 - global_seqlen/balanced_min:29583 - global_seqlen/balanced_max:29645 - global_seqlen/mean:29615.125 - actor/entropy:0.34415289759635925 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5622131824493408 - training/rollout_probs_diff_mean:0.003517036559060216 - training/rollout_probs_diff_std:0.009729686193168163 - training/rollout_actor_probs_pearson_corr:0.9993311762809753 - rollout_corr/rollout_is_mean:0.9999920129776001 - rollout_corr/rollout_is_ratio_fraction_high:1.2048701137246098e-05 - rollout_corr/rollout_is_ratio_fraction_low:3.0121751478873193e-05 - rollout_corr/rollout_is_max:8.23111629486084 - rollout_corr/rollout_is_min:0.2471064180135727 - rollout_corr/rollout_is_std:0.03159145638346672 - rollout_corr/rollout_is_eff_sample_size:0.9990024989924954 - rollout_corr/rollout_is_seq_mean:0.9999005794525146 - rollout_corr/rollout_is_seq_std:0.0016829889500513673 - rollout_corr/rollout_is_seq_max:1.004620909690857 - rollout_corr/rollout_is_seq_min:0.9928184747695923 - rollout_corr/rollout_is_seq_max_deviation:0.007181525230407715 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.372836111113429) - rollout_corr/training_log_ppl:np.float64(0.2916473155346466) - rollout_corr/kl:np.float64(0.0007178986236571205) - rollout_corr/k3_kl:np.float64(0.0006351841151968074) - rollout_corr/rollout_ppl:np.float64(1.3718020445667207) - rollout_corr/rollout_log_ppl:np.float64(0.2909294152777875) - rollout_corr/log_ppl_diff:np.float64(0.0007179002568591386) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014387322589755058) - rollout_corr/log_ppl_diff_max:np.float64(0.009183712303638458) - rollout_corr/log_ppl_diff_min:np.float64(-0.0071634650230407715) - rollout_corr/ppl_ratio:np.float64(1.0007200299296528) - rollout_corr/chi2_token:np.float64(0.0011453672777861357) - rollout_corr/chi2_seq:np.float64(0.5700722692999989) - actor/pg_loss:np.float64(0.00010708440095186234) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003794109481987107) - actor/ppo_kl:np.float64(5.79457945377726e-05) - actor/pg_clipfrac_lower:np.float64(4.787071247847052e-06) - actor/grad_norm:np.float64(0.27837222814559937) - perf/mfu/actor:np.float64(0.08514566987298042) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.15073013305664) - actor/lr:np.float64(1e-06) - training/global_step:83 - training/epoch:3 - critic/score/mean:0.66796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.003308121347799897 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.003308121347799897 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:648.41015625 - response_length/max:2853.0 - response_length/min:74.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:648.41015625 - response_length_non_aborted/max:2853.0 - response_length_non_aborted/min:74.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.0625 - prompt_length/max:375.0 - prompt_length/min:174.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:6.170570850372314e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8073275871574879) - timing_s/agent_loop/generate_sequences/max:np.float64(18.502459589391947) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.3145277952207834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(18.502459589391947) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:273 - timing_s/agent_loop/slowest/response_length:2853 - timing_s/gen:24.35667880065739 - timing_s/reward:0.10609085857868195 - timing_s/old_log_prob:2.4029372967779636 - timing_s/adv:0.11332383565604687 - timing_s/update_actor:10.112380102276802 - timing_s/step:37.18459020368755 - timing_s/stop_profile:3.2028183341026306e-05 - timing_per_token_ms/update_actor:0.042682497973066134 - timing_per_token_ms/gen:0.14673316827009206 - timing_per_token_ms/adv:0.0004783190838129455 - perf/total_num_tokens:236921 - perf/time_per_step:37.18459020368755 - perf/throughput:796.4354276267675 (TaskRunner pid=2049544) Training Progress: 83%|████████▎ | 83/100 [1:05:12<14:21, 50.66s/it] (TaskRunner pid=2049544) step:84 - global_seqlen/min:25752 - global_seqlen/max:42945 - global_seqlen/minmax_diff:17193 - global_seqlen/balanced_min:33932 - global_seqlen/balanced_max:34032 - global_seqlen/mean:33969.0 - actor/entropy:0.3499566912651062 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6138913631439209 - training/rollout_probs_diff_mean:0.0033732259180396795 - training/rollout_probs_diff_std:0.009146650321781635 - training/rollout_actor_probs_pearson_corr:0.9994776844978333 - rollout_corr/rollout_is_mean:0.9999618530273438 - rollout_corr/rollout_is_ratio_fraction_high:4.901384272670839e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.4506920453859493e-05 - rollout_corr/rollout_is_max:2.404789447784424 - rollout_corr/rollout_is_min:0.2902381718158722 - rollout_corr/rollout_is_std:0.03017878346145153 - rollout_corr/rollout_is_eff_sample_size:0.9990900697549667 - rollout_corr/rollout_is_seq_mean:1.0001087188720703 - rollout_corr/rollout_is_seq_std:0.001655240310356021 - rollout_corr/rollout_is_seq_max:1.0082566738128662 - rollout_corr/rollout_is_seq_min:0.9946045279502869 - rollout_corr/rollout_is_seq_max_deviation:0.008256673812866211 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3845429411157966) - rollout_corr/training_log_ppl:np.float64(0.29742183088092133) - rollout_corr/kl:np.float64(0.00039445633141355074) - rollout_corr/k3_kl:np.float64(0.0005463047522766828) - rollout_corr/rollout_ppl:np.float64(1.3839370189234614) - rollout_corr/rollout_log_ppl:np.float64(0.29702737282786984) - rollout_corr/log_ppl_diff:np.float64(0.00039445805305149406) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012936670245835558) - rollout_corr/log_ppl_diff_max:np.float64(0.00665166974067688) - rollout_corr/log_ppl_diff_min:np.float64(-0.005310043692588806) - rollout_corr/ppl_ratio:np.float64(1.000396022805944) - rollout_corr/chi2_token:np.float64(0.0013724297750741243) - rollout_corr/chi2_seq:np.float64(1.7039004366379231) - actor/pg_loss:np.float64(0.00012169475667178631) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00021371153513882746) - actor/ppo_kl:np.float64(1.5052716766383867e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.2066413126885891) - perf/mfu/actor:np.float64(0.09535977782956703) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.16190719604492) - actor/lr:np.float64(1e-06) - training/global_step:84 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017204470932483673 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017204470932483673 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:796.96875 - response_length/max:3594.0 - response_length/min:81.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:796.96875 - response_length_non_aborted/max:3594.0 - response_length_non_aborted/min:81.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.5625 - prompt_length/max:356.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.7237460017204285e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9937344156205654) - timing_s/agent_loop/generate_sequences/max:np.float64(23.47761689312756) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.72609610089421) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.47761689312756) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:356 - timing_s/agent_loop/slowest/response_length:3594 - timing_s/gen:29.4396906606853 - timing_s/reward:0.07473043724894524 - timing_s/old_log_prob:2.594569843262434 - timing_s/adv:0.12372583709657192 - timing_s/update_actor:10.489614827558398 - timing_s/step:42.81442074663937 - timing_s/stop_profile:2.9224902391433716e-05 - timing_per_token_ms/update_actor:0.038599954471571135 - timing_per_token_ms/gen:0.14429523321121682 - timing_per_token_ms/adv:0.000455289518003812 - perf/total_num_tokens:271752 - perf/time_per_step:42.81442074663937 - perf/throughput:793.4009010892977 (TaskRunner pid=2049544) Training Progress: 84%|████████▍ | 84/100 [1:05:55<12:52, 48.31s/it] (TaskRunner pid=2049544) step:85 - global_seqlen/min:20755 - global_seqlen/max:54433 - global_seqlen/minmax_diff:33678 - global_seqlen/balanced_min:33282 - global_seqlen/balanced_max:33493 - global_seqlen/mean:33418.125 - actor/entropy:0.29106423258781433 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4967511296272278 - training/rollout_probs_diff_mean:0.003082117298617959 - training/rollout_probs_diff_std:0.009240406565368176 - training/rollout_actor_probs_pearson_corr:0.9993290901184082 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:4.992785306967562e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.9971141227870248e-05 - rollout_corr/rollout_is_max:2.3519351482391357 - rollout_corr/rollout_is_min:0.26859748363494873 - rollout_corr/rollout_is_std:0.029412612318992615 - rollout_corr/rollout_is_eff_sample_size:0.9991358839466532 - rollout_corr/rollout_is_seq_mean:0.9999825954437256 - rollout_corr/rollout_is_seq_std:0.0015941840829327703 - rollout_corr/rollout_is_seq_max:1.0068665742874146 - rollout_corr/rollout_is_seq_min:0.9917056560516357 - rollout_corr/rollout_is_seq_max_deviation:0.008294343948364258 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2718443693593144) - rollout_corr/training_log_ppl:np.float64(0.22443578000093112) - rollout_corr/kl:np.float64(0.0005144908149414817) - rollout_corr/k3_kl:np.float64(0.0004981835849378058) - rollout_corr/rollout_ppl:np.float64(1.2711914060637355) - rollout_corr/rollout_log_ppl:np.float64(0.22392128901265096) - rollout_corr/log_ppl_diff:np.float64(0.0005144909882801585) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011518053579493426) - rollout_corr/log_ppl_diff_max:np.float64(0.009792208671569824) - rollout_corr/log_ppl_diff_min:np.float64(-0.005565732717514038) - rollout_corr/ppl_ratio:np.float64(1.000515874940902) - rollout_corr/chi2_token:np.float64(0.000967925414443016) - rollout_corr/chi2_seq:np.float64(1.0396802069153637) - actor/pg_loss:np.float64(-0.0001277067931368947) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000293373741897085) - actor/ppo_kl:np.float64(1.70920517623685e-05) - actor/pg_clipfrac_lower:np.float64(5.778476406703703e-06) - actor/grad_norm:np.float64(0.20695016533136368) - perf/mfu/actor:np.float64(0.09336689611238912) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.0907974243164) - actor/lr:np.float64(1e-06) - training/global_step:85 - training/epoch:3 - critic/score/mean:0.671875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.671875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.014600777067244053 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.014600777067244053 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:782.37890625 - response_length/max:4261.0 - response_length/min:69.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:782.37890625 - response_length_non_aborted/max:4261.0 - response_length_non_aborted/min:69.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:261.9375 - prompt_length/max:365.0 - prompt_length/min:183.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:2.2804364562034607e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7515706215053797) - timing_s/agent_loop/generate_sequences/max:np.float64(26.178023107349873) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.071110863718786) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.178023107349873) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:4261 - timing_s/gen:31.903686491772532 - timing_s/reward:0.07636177726089954 - timing_s/old_log_prob:2.5928792282938957 - timing_s/adv:0.12288173846900463 - timing_s/update_actor:10.525410840287805 - timing_s/step:45.31749305874109 - timing_s/stop_profile:0.00011776015162467957 - timing_per_token_ms/update_actor:0.039370142850204064 - timing_per_token_ms/gen:0.15928826092183063 - timing_per_token_ms/adv:0.0004596373168340707 - perf/total_num_tokens:267345 - perf/time_per_step:45.31749305874109 - perf/throughput:737.4221905144448 (TaskRunner pid=2049544) Training Progress: 85%|████████▌ | 85/100 [1:06:41<11:51, 47.43s/it] (TaskRunner pid=2049544) step:86 - global_seqlen/min:22104 - global_seqlen/max:49662 - global_seqlen/minmax_diff:27558 - global_seqlen/balanced_min:36725 - global_seqlen/balanced_max:36775 - global_seqlen/mean:36762.375 - actor/entropy:0.3210241496562958 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.24132895469665527 - training/rollout_probs_diff_mean:0.003310859901830554 - training/rollout_probs_diff_std:0.009120251052081585 - training/rollout_actor_probs_pearson_corr:0.9993811845779419 - rollout_corr/rollout_is_mean:0.9998584389686584 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:2.2271813577390276e-05 - rollout_corr/rollout_is_max:1.9132159948349 - rollout_corr/rollout_is_min:0.41626274585723877 - rollout_corr/rollout_is_std:0.029449068009853363 - rollout_corr/rollout_is_eff_sample_size:0.9991331468769348 - rollout_corr/rollout_is_seq_mean:0.9998147487640381 - rollout_corr/rollout_is_seq_std:0.0013719926355406642 - rollout_corr/rollout_is_seq_max:1.0041395425796509 - rollout_corr/rollout_is_seq_min:0.9918246865272522 - rollout_corr/rollout_is_seq_max_deviation:0.008175313472747803 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.318041059654206) - rollout_corr/training_log_ppl:np.float64(0.26210865678149275) - rollout_corr/kl:np.float64(0.0005536363921763154) - rollout_corr/k3_kl:np.float64(0.0004948169941201286) - rollout_corr/rollout_ppl:np.float64(1.317284729797393) - rollout_corr/rollout_log_ppl:np.float64(0.2615550198825076) - rollout_corr/log_ppl_diff:np.float64(0.0005536368989851326) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010979093494825065) - rollout_corr/log_ppl_diff_max:np.float64(0.0066015273332595825) - rollout_corr/log_ppl_diff_min:np.float64(-0.002927526831626892) - rollout_corr/ppl_ratio:np.float64(1.0005547057371587) - rollout_corr/chi2_token:np.float64(0.0008581595029681921) - rollout_corr/chi2_seq:np.float64(0.48298407695256174) - actor/pg_loss:np.float64(0.00010274478700011969) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012036708881169034) - actor/ppo_kl:np.float64(-9.579434273687681e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1470967587083578) - perf/mfu/actor:np.float64(0.1030748564778111) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.2135009765625) - actor/lr:np.float64(1e-06) - training/global_step:86 - training/epoch:3 - critic/score/mean:0.6328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.007343573961406946 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.007343573961406946 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:876.94921875 - response_length/max:3278.0 - response_length/min:120.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:876.94921875 - response_length_non_aborted/max:3278.0 - response_length_non_aborted/min:120.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:271.875 - prompt_length/max:365.0 - prompt_length/min:182.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010039471089839935 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4529647156596184) - timing_s/agent_loop/generate_sequences/max:np.float64(22.549825344234705) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.41551528852142) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.549825344234705) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:244 - timing_s/agent_loop/slowest/response_length:3278 - timing_s/gen:28.685686687007546 - timing_s/reward:0.07788657583296299 - timing_s/old_log_prob:2.5779322646558285 - timing_s/adv:0.1303798407316208 - timing_s/update_actor:10.655327502638102 - timing_s/step:42.21963391080499 - timing_s/stop_profile:0.00011177733540534973 - timing_per_token_ms/update_actor:0.036230410516996325 - timing_per_token_ms/gen:0.1277764564074118 - timing_per_token_ms/adv:0.0004433195649479284 - perf/total_num_tokens:294099 - perf/time_per_step:42.21963391080499 - perf/throughput:870.7412072228237 (TaskRunner pid=2049544) Training Progress: 86%|████████▌ | 86/100 [1:07:23<10:42, 45.88s/it] (TaskRunner pid=2049544) step:87 - global_seqlen/min:29440 - global_seqlen/max:47768 - global_seqlen/minmax_diff:18328 - global_seqlen/balanced_min:35517 - global_seqlen/balanced_max:36384 - global_seqlen/mean:35692.75 - actor/entropy:0.3466551899909973 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6292303800582886 - training/rollout_probs_diff_mean:0.003489995375275612 - training/rollout_probs_diff_std:0.00928904302418232 - training/rollout_actor_probs_pearson_corr:0.9994551539421082 - rollout_corr/rollout_is_mean:1.0000280141830444 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:4.662482751882635e-06 - rollout_corr/rollout_is_max:1.702067494392395 - rollout_corr/rollout_is_min:0.26140108704566956 - rollout_corr/rollout_is_std:0.03009967692196369 - rollout_corr/rollout_is_eff_sample_size:0.9990948294754013 - rollout_corr/rollout_is_seq_mean:1.0000789165496826 - rollout_corr/rollout_is_seq_std:0.001175415120087564 - rollout_corr/rollout_is_seq_max:1.0036451816558838 - rollout_corr/rollout_is_seq_min:0.9961031079292297 - rollout_corr/rollout_is_seq_max_deviation:0.0038968920707702637 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.384202916175127) - rollout_corr/training_log_ppl:np.float64(0.2980515792296501) - rollout_corr/kl:np.float64(0.00042411336519165843) - rollout_corr/k3_kl:np.float64(0.00048466515880818406) - rollout_corr/rollout_ppl:np.float64(1.3836216540075839) - rollout_corr/rollout_log_ppl:np.float64(0.2976274649699917) - rollout_corr/log_ppl_diff:np.float64(0.00042411425965838134) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010522096417844296) - rollout_corr/log_ppl_diff_max:np.float64(0.004691094160079956) - rollout_corr/log_ppl_diff_min:np.float64(-0.005741387605667114) - rollout_corr/ppl_ratio:np.float64(1.0004250849597156) - rollout_corr/chi2_token:np.float64(0.001095710089430213) - rollout_corr/chi2_seq:np.float64(1.8486225868109614) - actor/pg_loss:np.float64(7.82626448199153e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016853076499501185) - actor/ppo_kl:np.float64(3.361720549453773e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.18693841621279716) - perf/mfu/actor:np.float64(0.0996124497047341) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.18145751953125) - actor/lr:np.float64(1e-06) - training/global_step:87 - training/epoch:3 - critic/score/mean:0.6875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.006155060138553381 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.006155060138553381 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:837.8046875 - response_length/max:4550.0 - response_length/min:119.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:837.8046875 - response_length_non_aborted/max:4550.0 - response_length_non_aborted/min:119.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:277.59375 - prompt_length/max:364.0 - prompt_length/min:219.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001092962920665741 - timing_s/agent_loop/generate_sequences/min:np.float64(1.4342339634895325) - timing_s/agent_loop/generate_sequences/max:np.float64(28.061727035790682) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.031970594347513) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.061727035790682) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:239 - timing_s/agent_loop/slowest/response_length:4550 - timing_s/gen:33.89907926507294 - timing_s/reward:0.07478075660765171 - timing_s/old_log_prob:2.563651405274868 - timing_s/adv:0.12060143612325191 - timing_s/update_actor:10.524738831445575 - timing_s/step:47.28383187763393 - timing_s/stop_profile:0.00016075000166893005 - timing_per_token_ms/update_actor:0.036858811773558965 - timing_per_token_ms/gen:0.15805387622540745 - timing_per_token_ms/adv:0.00042235970933611136 - perf/total_num_tokens:285542 - perf/time_per_step:47.28383187763393 - perf/throughput:754.8616214601526 (TaskRunner pid=2049544) Training Progress: 87%|████████▋ | 87/100 [1:08:10<10:02, 46.32s/it] (TaskRunner pid=2049544) step:88 - global_seqlen/min:22456 - global_seqlen/max:46812 - global_seqlen/minmax_diff:24356 - global_seqlen/balanced_min:35428 - global_seqlen/balanced_max:35596 - global_seqlen/mean:35553.625 - actor/entropy:0.3236216604709625 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.21511626243591309 - training/rollout_probs_diff_mean:0.0032852201256901026 - training/rollout_probs_diff_std:0.009024620987474918 - training/rollout_actor_probs_pearson_corr:0.999454140663147 - rollout_corr/rollout_is_mean:1.0000885725021362 - rollout_corr/rollout_is_ratio_fraction_high:4.604963123711059e-06 - rollout_corr/rollout_is_ratio_fraction_low:4.604963123711059e-06 - rollout_corr/rollout_is_max:2.0104920864105225 - rollout_corr/rollout_is_min:0.23549965023994446 - rollout_corr/rollout_is_std:0.029910963028669357 - rollout_corr/rollout_is_eff_sample_size:0.999106252989479 - rollout_corr/rollout_is_seq_mean:1.0000383853912354 - rollout_corr/rollout_is_seq_std:0.0014727648813277483 - rollout_corr/rollout_is_seq_max:1.0041821002960205 - rollout_corr/rollout_is_seq_min:0.9893348813056946 - rollout_corr/rollout_is_seq_max_deviation:0.01066511869430542 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.338182927109301) - rollout_corr/training_log_ppl:np.float64(0.2705517132417299) - rollout_corr/kl:np.float64(0.0005242268059881816) - rollout_corr/k3_kl:np.float64(0.000508854052498009) - rollout_corr/rollout_ppl:np.float64(1.3374784495681524) - rollout_corr/rollout_log_ppl:np.float64(0.27002748603990767) - rollout_corr/log_ppl_diff:np.float64(0.0005242272018222138) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011741442867787555) - rollout_corr/log_ppl_diff_max:np.float64(0.013874843716621399) - rollout_corr/log_ppl_diff_min:np.float64(-0.0045333802700042725) - rollout_corr/ppl_ratio:np.float64(1.0005257281009108) - rollout_corr/chi2_token:np.float64(0.0009958110749721527) - rollout_corr/chi2_seq:np.float64(1.3633345067501068) - actor/pg_loss:np.float64(1.946359407156706e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00016658261313295952) - actor/ppo_kl:np.float64(7.023494554836418e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.13375208340585232) - perf/mfu/actor:np.float64(0.09886611268499279) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.15729522705078) - actor/lr:np.float64(1e-06) - training/global_step:88 - training/epoch:3 - critic/score/mean:0.66015625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.66015625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.02385486103594303 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.02385486103594303 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:848.26953125 - response_length/max:4442.0 - response_length/min:80.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:848.26953125 - response_length_non_aborted/max:4442.0 - response_length_non_aborted/min:80.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:262.78125 - prompt_length/max:375.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.980984032154083e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.990953391417861) - timing_s/agent_loop/generate_sequences/max:np.float64(25.68886986002326) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.6535395380997215) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(25.68886986002326) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:294 - timing_s/agent_loop/slowest/response_length:4442 - timing_s/gen:31.64418187364936 - timing_s/reward:0.07790577039122581 - timing_s/old_log_prob:2.6047119572758675 - timing_s/adv:0.11337987333536148 - timing_s/update_actor:10.512858772650361 - timing_s/step:45.041199665516615 - timing_s/stop_profile:0.00016839616000652313 - timing_per_token_ms/update_actor:0.036961276004381975 - timing_per_token_ms/gen:0.1457202939516081 - timing_per_token_ms/adv:0.00039862276116486536 - perf/total_num_tokens:284429 - perf/time_per_step:45.041199665516615 - perf/throughput:789.3578604483693 (TaskRunner pid=2049544) Training Progress: 88%|████████▊ | 88/100 [1:08:55<09:11, 45.95s/it] (TaskRunner pid=2049544) step:89 - global_seqlen/min:23526 - global_seqlen/max:43993 - global_seqlen/minmax_diff:20467 - global_seqlen/balanced_min:32615 - global_seqlen/balanced_max:32696 - global_seqlen/mean:32672.25 - actor/entropy:0.3472384810447693 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.7165358662605286 - training/rollout_probs_diff_mean:0.003400100627914071 - training/rollout_probs_diff_std:0.009368252009153366 - training/rollout_actor_probs_pearson_corr:0.9993821382522583 - rollout_corr/rollout_is_mean:0.9999688267707825 - rollout_corr/rollout_is_ratio_fraction_high:5.2063269322388805e-06 - rollout_corr/rollout_is_ratio_fraction_low:3.123795977444388e-05 - rollout_corr/rollout_is_max:2.0695598125457764 - rollout_corr/rollout_is_min:0.2369716465473175 - rollout_corr/rollout_is_std:0.03049706295132637 - rollout_corr/rollout_is_eff_sample_size:0.9990706743631297 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0016654012724757195 - rollout_corr/rollout_is_seq_max:1.007972240447998 - rollout_corr/rollout_is_seq_min:0.9951695799827576 - rollout_corr/rollout_is_seq_max_deviation:0.007972240447998047 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3569685937836766) - rollout_corr/training_log_ppl:np.float64(0.27856632304610685) - rollout_corr/kl:np.float64(0.0005558686453759343) - rollout_corr/k3_kl:np.float64(0.0005486493795388014) - rollout_corr/rollout_ppl:np.float64(1.3561613205820322) - rollout_corr/rollout_log_ppl:np.float64(0.27801045119122136) - rollout_corr/log_ppl_diff:np.float64(0.0005558718548854813) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011855626216856763) - rollout_corr/log_ppl_diff_max:np.float64(0.005568467080593109) - rollout_corr/log_ppl_diff_min:np.float64(-0.006579384207725525) - rollout_corr/ppl_ratio:np.float64(1.0005573213566095) - rollout_corr/chi2_token:np.float64(0.0010650090407580137) - rollout_corr/chi2_seq:np.float64(0.8179540059063584) - actor/pg_loss:np.float64(3.63390427082777e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00022707523396547913) - actor/ppo_kl:np.float64(6.390776004217003e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.3162117004394531) - perf/mfu/actor:np.float64(0.09205693492850642) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.16839599609375) - actor/lr:np.float64(1e-06) - training/global_step:89 - training/epoch:4 - critic/score/mean:0.79296875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.79296875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01907663233578205 - critic/advantages/max:0.75 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01907663233578205 - critic/returns/max:0.75 - critic/returns/min:-0.875 - response_length/mean:750.2890625 - response_length/max:3252.0 - response_length/min:118.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:750.2890625 - response_length_non_aborted/max:3252.0 - response_length_non_aborted/min:118.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:270.71875 - prompt_length/max:375.0 - prompt_length/min:185.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0003261677920818329 - timing_s/agent_loop/generate_sequences/min:np.float64(1.3473277054727077) - timing_s/agent_loop/generate_sequences/max:np.float64(20.700606567785144) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.879736434209917) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(20.700606567785144) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:3252 - timing_s/gen:26.722555236890912 - timing_s/reward:0.0684677753597498 - timing_s/old_log_prob:2.669492883607745 - timing_s/adv:0.1653036493808031 - timing_s/update_actor:10.382515193894506 - timing_s/step:40.15039100125432 - timing_s/stop_profile:0.00012563727796077728 - timing_per_token_ms/update_actor:0.03972222296403869 - timing_per_token_ms/gen:0.13912635357669914 - timing_per_token_ms/adv:0.0006324313805324209 - perf/total_num_tokens:261378 - perf/time_per_step:40.15039100125432 - perf/throughput:813.7467453051529 (TaskRunner pid=2049544) Training Progress: 89%|████████▉ | 89/100 [1:09:36<08:07, 44.35s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 90} (TaskRunner pid=2049544) (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \text{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \text{cm} = 0.50 \, \text{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \text{N m}^2/\text{C}^2 $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We are to find the charge $ q $ using the formula: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Rearranging to solve for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the known values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \text{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \text{C} = 55.6 \, \text{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_80/actor (TaskRunner pid=2049544) step:90 - global_seqlen/min:22189 - global_seqlen/max:43070 - global_seqlen/minmax_diff:20881 - global_seqlen/balanced_min:29230 - global_seqlen/balanced_max:29298 - global_seqlen/mean:29287.75 - actor/entropy:0.3101608455181122 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3454935550689697 - training/rollout_probs_diff_mean:0.0033272632863372564 - training/rollout_probs_diff_std:0.009299682453274727 - training/rollout_actor_probs_pearson_corr:0.999302327632904 - rollout_corr/rollout_is_mean:0.9999826550483704 - rollout_corr/rollout_is_ratio_fraction_high:1.2050661098328419e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.2050661098328419e-05 - rollout_corr/rollout_is_max:2.8188319206237793 - rollout_corr/rollout_is_min:0.3406546115875244 - rollout_corr/rollout_is_std:0.03000447526574135 - rollout_corr/rollout_is_eff_sample_size:0.9991005411997865 - rollout_corr/rollout_is_seq_mean:1.0000338554382324 - rollout_corr/rollout_is_seq_std:0.0013818239094689488 - rollout_corr/rollout_is_seq_max:1.0056010484695435 - rollout_corr/rollout_is_seq_min:0.9940690398216248 - rollout_corr/rollout_is_seq_max_deviation:0.005930960178375244 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3156746202148497) - rollout_corr/training_log_ppl:np.float64(0.2531014566193335) - rollout_corr/kl:np.float64(0.0005285690323812986) - rollout_corr/k3_kl:np.float64(0.0004731722787880699) - rollout_corr/rollout_ppl:np.float64(1.3148961896076798) - rollout_corr/rollout_log_ppl:np.float64(0.25257288754801266) - rollout_corr/log_ppl_diff:np.float64(0.0005285690713208169) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010851562838070095) - rollout_corr/log_ppl_diff_max:np.float64(0.006567090749740601) - rollout_corr/log_ppl_diff_min:np.float64(-0.00448918342590332) - rollout_corr/ppl_ratio:np.float64(1.0005297421012074) - rollout_corr/chi2_token:np.float64(0.0008521552663296461) - rollout_corr/chi2_seq:np.float64(0.22603251412510872) - actor/pg_loss:np.float64(0.00011028861626982689) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00012377322673273738) - actor/ppo_kl:np.float64(0.00010799788028137414) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.14029540959745646) - perf/mfu/actor:np.float64(0.08421706394502831) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.20226669311523) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6625) - val-aux/sciknoweval/reward/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.10800278634919543) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8915625) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.04701965592013402) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.693225) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.0730352339040406) - val-aux/sciknoweval/score/mean@16:np.float64(0.6625) - val-aux/sciknoweval/score/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/score/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/score/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.10800278634919543) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8915625) - val-aux/sciknoweval/score/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.04701965592013402) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.693225) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.0730352339040406) - val-core/sciknoweval/acc/mean@16:np.float64(0.6625) - val-aux/sciknoweval/acc/std@16:np.float64(0.18997502251966206) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.739025) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1632991687244558) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.587875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15720634779503662) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6636) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.1893282012924375) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.8018125000000002) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.12815819440702003) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5259125000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.1162500052562232) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.6786875) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.1497440626684113) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8526875) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.09374063114057554) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4800625) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.07881481778772832) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.688175) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.10800278634919543) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8915625) - val-core/sciknoweval/acc/best@16/std:np.float64(0.06088379895875816) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.449325) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.04701965592013402) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.693225) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.0730352339040406) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9890625) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03660810518441162) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.9990500000000001) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.010677570009542227) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9789875) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.048719255966368245) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.989175) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.036500484252753186) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9618874999999999) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.06036258960895739) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9962249999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021651315045931742) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.9349000000000001) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.06689717910953397) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9992000000000001) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.009789340111118103) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9021374999999999) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.05987592930423645) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9999375) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0017437187447881502) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:90 - training/epoch:4 - critic/score/mean:0.7421875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7421875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.01157616637647152 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.01157616637647152 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:648.3046875 - response_length/max:2634.0 - response_length/min:157.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:648.3046875 - response_length_non_aborted/max:2634.0 - response_length_non_aborted/min:157.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:266.9375 - prompt_length/max:437.0 - prompt_length/min:210.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011180341243743896 - timing_s/agent_loop/generate_sequences/min:np.float64(1.8763771429657936) - timing_s/agent_loop/generate_sequences/max:np.float64(17.981070213019848) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.232629431753594) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(17.981070213019848) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:320 - timing_s/agent_loop/slowest/response_length:2634 - timing_s/gen:23.51352482289076 - timing_s/reward:0.0758508276194334 - timing_s/old_log_prob:2.507504915818572 - timing_s/adv:0.1161189116537571 - timing_s/update_actor:10.261307056993246 - timing_s/step:36.5611352995038 - timing_s/testing:112.82421230524778 - timing_s/save_checkpoint:6.677826223894954 - timing_s/stop_profile:0.0001322384923696518 - timing_per_token_ms/update_actor:0.04379521752692357 - timing_per_token_ms/gen:0.14167675802809465 - timing_per_token_ms/adv:0.0004955950510612675 - perf/total_num_tokens:234302 - perf/time_per_step:36.5611352995038 - perf/throughput:801.0623783993241 (TaskRunner pid=2049544) Training Progress: 90%|█████████ | 90/100 [1:12:12<12:58, 77.88s/it] (TaskRunner pid=2049544) step:91 - global_seqlen/min:21497 - global_seqlen/max:45454 - global_seqlen/minmax_diff:23957 - global_seqlen/balanced_min:32904 - global_seqlen/balanced_max:33139 - global_seqlen/mean:33002.875 - actor/entropy:0.26041534543037415 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.34585317969322205 - training/rollout_probs_diff_mean:0.002946822438389063 - training/rollout_probs_diff_std:0.008950787596404552 - training/rollout_actor_probs_pearson_corr:0.9992419481277466 - rollout_corr/rollout_is_mean:0.9999063611030579 - rollout_corr/rollout_is_ratio_fraction_high:5.014869202568661e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.5044607607705984e-05 - rollout_corr/rollout_is_max:5.096095561981201 - rollout_corr/rollout_is_min:0.37255313992500305 - rollout_corr/rollout_is_std:0.028246553614735603 - rollout_corr/rollout_is_eff_sample_size:0.9992028873298154 - rollout_corr/rollout_is_seq_mean:1.0000227689743042 - rollout_corr/rollout_is_seq_std:0.001477584009990096 - rollout_corr/rollout_is_seq_max:1.0067633390426636 - rollout_corr/rollout_is_seq_min:0.995822548866272 - rollout_corr/rollout_is_seq_max_deviation:0.006763339042663574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2352091469801962) - rollout_corr/training_log_ppl:np.float64(0.20221140759531409) - rollout_corr/kl:np.float64(0.00045496249386900445) - rollout_corr/k3_kl:np.float64(0.00047493476955651204) - rollout_corr/rollout_ppl:np.float64(1.2346553695388138) - rollout_corr/rollout_log_ppl:np.float64(0.2017564445413882) - rollout_corr/log_ppl_diff:np.float64(0.0004549630539258942) - rollout_corr/log_ppl_abs_diff:np.float64(0.001167977330624126) - rollout_corr/log_ppl_diff_max:np.float64(0.009916573762893677) - rollout_corr/log_ppl_diff_min:np.float64(-0.006213173270225525) - rollout_corr/ppl_ratio:np.float64(1.0004563292022794) - rollout_corr/chi2_token:np.float64(0.0009643365629017353) - rollout_corr/chi2_seq:np.float64(0.6764775053597987) - actor/pg_loss:np.float64(5.630298983305693e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001349058953792337) - actor/ppo_kl:np.float64(3.4980036296872186e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12929855473339558) - perf/mfu/actor:np.float64(0.09015558565254489) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.73992156982422) - actor/lr:np.float64(1e-06) - training/global_step:91 - training/epoch:4 - critic/score/mean:0.81640625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.81640625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.012762214988470078 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.012762214988470078 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:778.93359375 - response_length/max:5011.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:778.93359375 - response_length_non_aborted/max:5011.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:252.40625 - prompt_length/max:340.0 - prompt_length/min:166.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.5048614740371704e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(0.9828272927552462) - timing_s/agent_loop/generate_sequences/max:np.float64(30.033628037199378) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.231723692668311) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(30.033628037199378) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:242 - timing_s/agent_loop/slowest/response_length:5011 - timing_s/gen:35.68446674197912 - timing_s/reward:0.0753241665661335 - timing_s/old_log_prob:2.7556536067277193 - timing_s/adv:0.12121403589844704 - timing_s/update_actor:10.860587490722537 - timing_s/step:49.55868074670434 - timing_s/stop_profile:0.00011924095451831818 - timing_per_token_ms/update_actor:0.04113500524849175 - timing_per_token_ms/gen:0.17895292914480998 - timing_per_token_ms/adv:0.0004591040776691691 - perf/total_num_tokens:264023 - perf/time_per_step:49.55868074670434 - perf/throughput:665.9353013991337 (TaskRunner pid=2049544) Training Progress: 91%|█████████ | 91/100 [1:13:02<10:24, 69.39s/it] (TaskRunner pid=2049544) step:92 - global_seqlen/min:27995 - global_seqlen/max:54733 - global_seqlen/minmax_diff:26738 - global_seqlen/balanced_min:38799 - global_seqlen/balanced_max:39948 - global_seqlen/mean:39032.875 - actor/entropy:0.365276038646698 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.4553599953651428 - training/rollout_probs_diff_mean:0.0034760674461722374 - training/rollout_probs_diff_std:0.009343450888991356 - training/rollout_actor_probs_pearson_corr:0.999370813369751 - rollout_corr/rollout_is_mean:0.9999127984046936 - rollout_corr/rollout_is_ratio_fraction_high:8.297963177028578e-06 - rollout_corr/rollout_is_ratio_fraction_low:2.0744908397318795e-05 - rollout_corr/rollout_is_max:3.3402388095855713 - rollout_corr/rollout_is_min:0.3921363651752472 - rollout_corr/rollout_is_std:0.03065885230898857 - rollout_corr/rollout_is_eff_sample_size:0.999060798471362 - rollout_corr/rollout_is_seq_mean:0.9999949932098389 - rollout_corr/rollout_is_seq_std:0.001438560662791133 - rollout_corr/rollout_is_seq_max:1.005057454109192 - rollout_corr/rollout_is_seq_min:0.9950283765792847 - rollout_corr/rollout_is_seq_max_deviation:0.0050574541091918945 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4253517426550388) - rollout_corr/training_log_ppl:np.float64(0.3268485037915525) - rollout_corr/kl:np.float64(0.0007143817989492263) - rollout_corr/k3_kl:np.float64(0.0007667658379375553) - rollout_corr/rollout_ppl:np.float64(1.4242633627727628) - rollout_corr/rollout_log_ppl:np.float64(0.3261341213074047) - rollout_corr/log_ppl_diff:np.float64(0.0007143824841477908) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013714263404835947) - rollout_corr/log_ppl_diff_max:np.float64(0.013571739196777344) - rollout_corr/log_ppl_diff_min:np.float64(-0.0037727057933807373) - rollout_corr/ppl_ratio:np.float64(1.000716715119779) - rollout_corr/chi2_token:np.float64(0.001422470435500145) - rollout_corr/chi2_seq:np.float64(0.832664096262306) - actor/pg_loss:np.float64(2.3467233404517174e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0003434544945548623) - actor/ppo_kl:np.float64(0.00016380211024369373) - actor/pg_clipfrac_lower:np.float64(2.0755844616360264e-06) - actor/grad_norm:np.float64(0.215729471296072) - perf/mfu/actor:np.float64(0.10647986423451218) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.4896354675293) - actor/lr:np.float64(1e-06) - training/global_step:92 - training/epoch:4 - critic/score/mean:0.69921875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.69921875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.011413848027586937 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.011413848027586937 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:941.49609375 - response_length/max:5473.0 - response_length/min:148.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:941.49609375 - response_length_non_aborted/max:5473.0 - response_length_non_aborted/min:148.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:278.28125 - prompt_length/max:460.0 - prompt_length/min:169.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:5.439482629299164e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.7557445913553238) - timing_s/agent_loop/generate_sequences/max:np.float64(32.87127552181482) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.810325469457894) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(32.87127552181482) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:219 - timing_s/agent_loop/slowest/response_length:5473 - timing_s/gen:38.770385617390275 - timing_s/reward:0.08092910423874855 - timing_s/old_log_prob:2.7198645491153 - timing_s/adv:0.12597152031958103 - timing_s/update_actor:10.861796030774713 - timing_s/step:52.657433899119496 - timing_s/stop_profile:0.00011220574378967285 - timing_per_token_ms/update_actor:0.034784127580836384 - timing_per_token_ms/gen:0.1608576178098782 - timing_per_token_ms/adv:0.00040341481481821745 - perf/total_num_tokens:312263 - perf/time_per_step:52.657433899119496 - perf/throughput:741.2604851724968 (TaskRunner pid=2049544) Training Progress: 92%|█████████▏| 92/100 [1:13:54<08:35, 64.39s/it] (TaskRunner pid=2049544) step:93 - global_seqlen/min:17843 - global_seqlen/max:51915 - global_seqlen/minmax_diff:34072 - global_seqlen/balanced_min:30929 - global_seqlen/balanced_max:31113 - global_seqlen/mean:31065.0 - actor/entropy:0.4133561849594116 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.5311214327812195 - training/rollout_probs_diff_mean:0.0036772009916603565 - training/rollout_probs_diff_std:0.009407786652445793 - training/rollout_actor_probs_pearson_corr:0.9994404911994934 - rollout_corr/rollout_is_mean:0.9999762177467346 - rollout_corr/rollout_is_ratio_fraction_high:5.474292720464291e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.6422878616140224e-05 - rollout_corr/rollout_is_max:2.111367702484131 - rollout_corr/rollout_is_min:0.21797895431518555 - rollout_corr/rollout_is_std:0.031457215547561646 - rollout_corr/rollout_is_eff_sample_size:0.9990114219410976 - rollout_corr/rollout_is_seq_mean:1.0000097751617432 - rollout_corr/rollout_is_seq_std:0.0018004179000854492 - rollout_corr/rollout_is_seq_max:1.007155179977417 - rollout_corr/rollout_is_seq_min:0.9935778975486755 - rollout_corr/rollout_is_seq_max_deviation:0.007155179977416992 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.4254543129354715) - rollout_corr/training_log_ppl:np.float64(0.3189181691413978) - rollout_corr/kl:np.float64(0.0006560000657124254) - rollout_corr/k3_kl:np.float64(0.0006174342987037562) - rollout_corr/rollout_ppl:np.float64(1.4244641666300595) - rollout_corr/rollout_log_ppl:np.float64(0.31826216776971705) - rollout_corr/log_ppl_diff:np.float64(0.0006560013716807589) - rollout_corr/log_ppl_abs_diff:np.float64(0.0014309252292150632) - rollout_corr/log_ppl_diff_max:np.float64(0.00973115861415863) - rollout_corr/log_ppl_diff_min:np.float64(-0.004590287804603577) - rollout_corr/ppl_ratio:np.float64(1.0006579444743693) - rollout_corr/chi2_token:np.float64(0.0011531137861311436) - rollout_corr/chi2_seq:np.float64(0.8776039613876492) - actor/pg_loss:np.float64(0.00012877536937594414) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000180150422920633) - actor/ppo_kl:np.float64(9.64217530006195e-05) - actor/pg_clipfrac_lower:np.float64(3.570612534531392e-06) - actor/grad_norm:np.float64(0.18000993877649307) - perf/mfu/actor:np.float64(0.08642174822282235) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.50779724121094) - actor/lr:np.float64(1e-06) - training/global_step:93 - training/epoch:4 - critic/score/mean:0.734375 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.734375 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.016780760139226913 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.016780760139226913 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:713.5625 - response_length/max:4647.0 - response_length/min:97.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:713.5625 - response_length_non_aborted/max:4647.0 - response_length_non_aborted/min:97.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:257.21875 - prompt_length/max:375.0 - prompt_length/min:160.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010390952229499817 - timing_s/agent_loop/generate_sequences/min:np.float64(0.7623797114938498) - timing_s/agent_loop/generate_sequences/max:np.float64(27.237358892336488) - timing_s/agent_loop/generate_sequences/mean:np.float64(5.646888445196964) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(27.237358892336488) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:259 - timing_s/agent_loop/slowest/response_length:4647 - timing_s/gen:33.228857627138495 - timing_s/reward:0.07251898013055325 - timing_s/old_log_prob:2.6381760984659195 - timing_s/adv:0.12340777926146984 - timing_s/update_actor:10.632954278960824 - timing_s/step:46.78136018849909 - timing_s/stop_profile:0.0001156739890575409 - timing_per_token_ms/update_actor:0.04278510493707075 - timing_per_token_ms/gen:0.18190449344802978 - timing_per_token_ms/adv:0.0004965708162782466 - perf/total_num_tokens:248520 - perf/time_per_step:46.78136018849909 - perf/throughput:664.0465320979944 (TaskRunner pid=2049544) Training Progress: 93%|█████████▎| 93/100 [1:14:41<06:53, 59.12s/it] (TaskRunner pid=2049544) step:94 - global_seqlen/min:22853 - global_seqlen/max:49175 - global_seqlen/minmax_diff:26322 - global_seqlen/balanced_min:33215 - global_seqlen/balanced_max:33359 - global_seqlen/mean:33330.0 - actor/entropy:0.36058807373046875 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.8867406845092773 - training/rollout_probs_diff_mean:0.003440327476710081 - training/rollout_probs_diff_std:0.009632454253733158 - training/rollout_actor_probs_pearson_corr:0.9993698000907898 - rollout_corr/rollout_is_mean:0.9999211430549622 - rollout_corr/rollout_is_ratio_fraction_high:1.0051463505078573e-05 - rollout_corr/rollout_is_ratio_fraction_low:4.523158713709563e-05 - rollout_corr/rollout_is_max:2.117000102996826 - rollout_corr/rollout_is_min:0.0013813151745125651 - rollout_corr/rollout_is_std:0.030682172626256943 - rollout_corr/rollout_is_eff_sample_size:0.9990594896328889 - rollout_corr/rollout_is_seq_mean:0.9999591112136841 - rollout_corr/rollout_is_seq_std:0.0020710721146315336 - rollout_corr/rollout_is_seq_max:1.011606216430664 - rollout_corr/rollout_is_seq_min:0.9853842854499817 - rollout_corr/rollout_is_seq_max_deviation:0.01461571455001831 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3442378360778093) - rollout_corr/training_log_ppl:np.float64(0.2689030548208393) - rollout_corr/kl:np.float64(0.0007270200237998736) - rollout_corr/k3_kl:np.float64(0.0005578451756775848) - rollout_corr/rollout_ppl:np.float64(1.3432344142347574) - rollout_corr/rollout_log_ppl:np.float64(0.26817603285599034) - rollout_corr/log_ppl_diff:np.float64(0.0007270219648489729) - rollout_corr/log_ppl_abs_diff:np.float64(0.0013523521920433268) - rollout_corr/log_ppl_diff_max:np.float64(0.018366605043411255) - rollout_corr/log_ppl_diff_min:np.float64(-0.00493505597114563) - rollout_corr/ppl_ratio:np.float64(1.0007292504888028) - rollout_corr/chi2_token:np.float64(0.0007120152004063129) - rollout_corr/chi2_seq:np.float64(0.3402676989790052) - actor/pg_loss:np.float64(-4.822376649826765e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(9.676928118551587e-05) - actor/ppo_kl:np.float64(0.00014517965372551345) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16034816205501556) - perf/mfu/actor:np.float64(0.09308032359557787) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.54820251464844) - actor/lr:np.float64(1e-06) - training/global_step:94 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0465502105653286 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0465502105653286 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:777.25 - response_length/max:3614.0 - response_length/min:72.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:777.25 - response_length_non_aborted/max:3614.0 - response_length_non_aborted/min:72.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:264.3125 - prompt_length/max:341.0 - prompt_length/min:155.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.0001496635377407074 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8505959454923868) - timing_s/agent_loop/generate_sequences/max:np.float64(23.250541422516108) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.11456365948834) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(23.250541422516108) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:293 - timing_s/agent_loop/slowest/response_length:3614 - timing_s/gen:29.206484995782375 - timing_s/reward:0.07558401487767696 - timing_s/old_log_prob:2.6541784573346376 - timing_s/adv:0.12619186379015446 - timing_s/update_actor:10.563141915947199 - timing_s/step:42.71316540054977 - timing_s/stop_profile:0.00011851266026496887 - timing_per_token_ms/update_actor:0.03961574375917792 - timing_per_token_ms/gen:0.14678395884821474 - timing_per_token_ms/adv:0.0004732668158946687 - perf/total_num_tokens:266640 - perf/time_per_step:42.71316540054977 - perf/throughput:780.3214696790185 (TaskRunner pid=2049544) Training Progress: 94%|█████████▍| 94/100 [1:15:24<05:25, 54.21s/it] (TaskRunner pid=2049544) step:95 - global_seqlen/min:27066 - global_seqlen/max:58256 - global_seqlen/minmax_diff:31190 - global_seqlen/balanced_min:37178 - global_seqlen/balanced_max:37249 - global_seqlen/mean:37221.375 - actor/entropy:0.3698161244392395 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.2553989887237549 - training/rollout_probs_diff_mean:0.0034687381703406572 - training/rollout_probs_diff_std:0.009108307771384716 - training/rollout_actor_probs_pearson_corr:0.999431848526001 - rollout_corr/rollout_is_mean:0.9999311566352844 - rollout_corr/rollout_is_ratio_fraction_high:0.0 - rollout_corr/rollout_is_ratio_fraction_low:1.3214753380452748e-05 - rollout_corr/rollout_is_max:1.8554537296295166 - rollout_corr/rollout_is_min:0.4261438846588135 - rollout_corr/rollout_is_std:0.030212340876460075 - rollout_corr/rollout_is_eff_sample_size:0.9990878089036473 - rollout_corr/rollout_is_seq_mean:0.9999582767486572 - rollout_corr/rollout_is_seq_std:0.0013638396048918366 - rollout_corr/rollout_is_seq_max:1.0052945613861084 - rollout_corr/rollout_is_seq_min:0.9958161115646362 - rollout_corr/rollout_is_seq_max_deviation:0.0052945613861083984 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3786341231316328) - rollout_corr/training_log_ppl:np.float64(0.30179976880026516) - rollout_corr/kl:np.float64(0.0004993855567150263) - rollout_corr/k3_kl:np.float64(0.0005308137937447555) - rollout_corr/rollout_ppl:np.float64(1.377938257995993) - rollout_corr/rollout_log_ppl:np.float64(0.30130038285278715) - rollout_corr/log_ppl_diff:np.float64(0.0004993859474780038) - rollout_corr/log_ppl_abs_diff:np.float64(0.001156414728029631) - rollout_corr/log_ppl_diff_max:np.float64(0.0048898980021476746) - rollout_corr/log_ppl_diff_min:np.float64(-0.00559312105178833) - rollout_corr/ppl_ratio:np.float64(1.0005006100982428) - rollout_corr/chi2_token:np.float64(0.001144631765782833) - rollout_corr/chi2_seq:np.float64(0.9150484425481409) - actor/pg_loss:np.float64(-2.8662499971687794e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.000206823247367538) - actor/ppo_kl:np.float64(-2.0183409755958337e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1875382363796234) - perf/mfu/actor:np.float64(0.10496417728509784) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.40451049804688) - actor/lr:np.float64(1e-06) - training/global_step:95 - training/epoch:4 - critic/score/mean:0.625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.021443139761686325 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.021443139761686325 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:886.79296875 - response_length/max:3209.0 - response_length/min:125.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:886.79296875 - response_length_non_aborted/max:3209.0 - response_length_non_aborted/min:125.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:276.375 - prompt_length/max:375.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:9.542517364025116e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.471508091315627) - timing_s/agent_loop/generate_sequences/max:np.float64(22.023407490924) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.383664465822221) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.023407490924) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:277 - timing_s/agent_loop/slowest/response_length:3175 - timing_s/gen:28.187608854845166 - timing_s/reward:0.08735943771898746 - timing_s/old_log_prob:2.6067604515701532 - timing_s/adv:0.13234028778970242 - timing_s/update_actor:10.463114799931645 - timing_s/step:41.565891321748495 - timing_s/stop_profile:0.00011608004570007324 - timing_per_token_ms/update_actor:0.035138125606360746 - timing_per_token_ms/gen:0.12416409575782276 - timing_per_token_ms/adv:0.00044443645549668175 - perf/total_num_tokens:297771 - perf/time_per_step:41.565891321748495 - perf/throughput:895.4788124686426 (TaskRunner pid=2049544) Training Progress: 95%|█████████▌| 95/100 [1:16:06<04:12, 50.43s/it] (TaskRunner pid=2049544) step:96 - global_seqlen/min:20271 - global_seqlen/max:71208 - global_seqlen/minmax_diff:50937 - global_seqlen/balanced_min:37576 - global_seqlen/balanced_max:37647 - global_seqlen/mean:37613.5 - actor/entropy:0.334806889295578 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.31711164116859436 - training/rollout_probs_diff_mean:0.003466662485152483 - training/rollout_probs_diff_std:0.009574675001204014 - training/rollout_actor_probs_pearson_corr:0.9992694854736328 - rollout_corr/rollout_is_mean:1.0000592470169067 - rollout_corr/rollout_is_ratio_fraction_high:8.720068763068411e-06 - rollout_corr/rollout_is_ratio_fraction_low:8.720068763068411e-06 - rollout_corr/rollout_is_max:2.086963415145874 - rollout_corr/rollout_is_min:0.4454006552696228 - rollout_corr/rollout_is_std:0.030977971851825714 - rollout_corr/rollout_is_eff_sample_size:0.9990414042159874 - rollout_corr/rollout_is_seq_mean:1.0001949071884155 - rollout_corr/rollout_is_seq_std:0.0014157574623823166 - rollout_corr/rollout_is_seq_max:1.0063108205795288 - rollout_corr/rollout_is_seq_min:0.9960592985153198 - rollout_corr/rollout_is_seq_max_deviation:0.006310820579528809 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3516691238619387) - rollout_corr/training_log_ppl:np.float64(0.27739123623177875) - rollout_corr/kl:np.float64(0.00038778805755357304) - rollout_corr/k3_kl:np.float64(0.0005165411663199393) - rollout_corr/rollout_ppl:np.float64(1.3510792162269354) - rollout_corr/rollout_log_ppl:np.float64(0.277003448762116) - rollout_corr/log_ppl_diff:np.float64(0.0003877874696627259) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011570610222406685) - rollout_corr/log_ppl_diff_max:np.float64(0.00403740257024765) - rollout_corr/log_ppl_diff_min:np.float64(-0.005538493394851685) - rollout_corr/ppl_ratio:np.float64(1.000388921238482) - rollout_corr/chi2_token:np.float64(0.0012935972772538662) - rollout_corr/chi2_seq:np.float64(2.2381823365576565) - actor/pg_loss:np.float64(5.385885015130043e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00015849828230329877) - actor/ppo_kl:np.float64(8.225317838395085e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.11362059134989977) - perf/mfu/actor:np.float64(0.1031450502424441) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.59148025512695) - actor/lr:np.float64(1e-06) - training/global_step:96 - training/epoch:4 - critic/score/mean:0.67578125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.67578125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0007041455246508121 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:0.0007041455246508121 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:895.921875 - response_length/max:4074.0 - response_length/min:184.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:895.921875 - response_length_non_aborted/max:4074.0 - response_length_non_aborted/min:184.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:279.5 - prompt_length/max:355.0 - prompt_length/min:200.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:7.657893002033234e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(2.2360860873013735) - timing_s/agent_loop/generate_sequences/max:np.float64(26.039900217205286) - timing_s/agent_loop/generate_sequences/mean:np.float64(8.459900658301194) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(26.039900217205286) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:314 - timing_s/agent_loop/slowest/response_length:3982 - timing_s/gen:32.00374800339341 - timing_s/reward:0.0823327787220478 - timing_s/old_log_prob:2.7716220505535603 - timing_s/adv:0.1664755642414093 - timing_s/update_actor:10.801547262817621 - timing_s/step:45.91564348526299 - timing_s/stop_profile:0.00011668726801872253 - timing_per_token_ms/update_actor:0.035896510770127815 - timing_per_token_ms/gen:0.13953743526828777 - timing_per_token_ms/adv:0.0005532440621100446 - perf/total_num_tokens:300908 - perf/time_per_step:45.91564348526299 - perf/throughput:819.1870383363432 (TaskRunner pid=2049544) Training Progress: 96%|█████████▌| 96/100 [1:16:52<03:16, 49.09s/it] (TaskRunner pid=2049544) step:97 - global_seqlen/min:20006 - global_seqlen/max:63623 - global_seqlen/minmax_diff:43617 - global_seqlen/balanced_min:40703 - global_seqlen/balanced_max:40790 - global_seqlen/mean:40767.5 - actor/entropy:0.4114430248737335 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.3968365788459778 - training/rollout_probs_diff_mean:0.0036807965952903032 - training/rollout_probs_diff_std:0.009112636558711529 - training/rollout_actor_probs_pearson_corr:0.9994710087776184 - rollout_corr/rollout_is_mean:0.9999697208404541 - rollout_corr/rollout_is_ratio_fraction_high:3.885426394845126e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.5541705579380505e-05 - rollout_corr/rollout_is_max:6.432518005371094 - rollout_corr/rollout_is_min:0.14259245991706848 - rollout_corr/rollout_is_std:0.030860379338264465 - rollout_corr/rollout_is_eff_sample_size:0.9990483051530745 - rollout_corr/rollout_is_seq_mean:1.000016450881958 - rollout_corr/rollout_is_seq_std:0.0015233448939397931 - rollout_corr/rollout_is_seq_max:1.0063254833221436 - rollout_corr/rollout_is_seq_min:0.994326114654541 - rollout_corr/rollout_is_seq_max_deviation:0.006325483322143555 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3867830368690193) - rollout_corr/training_log_ppl:np.float64(0.30247966502793133) - rollout_corr/kl:np.float64(0.0005262127845906406) - rollout_corr/k3_kl:np.float64(0.0005436497510515892) - rollout_corr/rollout_ppl:np.float64(1.386001723818481) - rollout_corr/rollout_log_ppl:np.float64(0.30195345119864214) - rollout_corr/log_ppl_diff:np.float64(0.0005262138292891905) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011905098654096946) - rollout_corr/log_ppl_diff_max:np.float64(0.007206156849861145) - rollout_corr/log_ppl_diff_min:np.float64(-0.003447234630584717) - rollout_corr/ppl_ratio:np.float64(1.0005276133306324) - rollout_corr/chi2_token:np.float64(0.001151977339759469) - rollout_corr/chi2_seq:np.float64(0.950474871089682) - actor/pg_loss:np.float64(6.92273024469614e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00019685469499108876) - actor/ppo_kl:np.float64(3.492400511362348e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.16588949039578438) - perf/mfu/actor:np.float64(0.11018383492360453) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.53912734985352) - actor/lr:np.float64(1e-06) - training/global_step:97 - training/epoch:4 - critic/score/mean:0.6796875 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.6796875 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.043718818575143814 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.043718818575143814 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:1005.359375 - response_length/max:4322.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:1005.359375 - response_length_non_aborted/max:4322.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:268.625 - prompt_length/max:353.0 - prompt_length/min:163.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00012983568012714386 - timing_s/agent_loop/generate_sequences/min:np.float64(1.021046968176961) - timing_s/agent_loop/generate_sequences/max:np.float64(28.757525622844696) - timing_s/agent_loop/generate_sequences/mean:np.float64(9.163475782763271) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(28.757525622844696) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:353 - timing_s/agent_loop/slowest/response_length:4322 - timing_s/gen:34.78881582431495 - timing_s/reward:0.08717712201178074 - timing_s/old_log_prob:2.7238584086298943 - timing_s/adv:0.1590470802038908 - timing_s/update_actor:10.954232659190893 - timing_s/step:48.80308444797993 - timing_s/stop_profile:0.00013154186308383942 - timing_per_token_ms/update_actor:0.0335875165854875 - timing_per_token_ms/gen:0.13516938837291917 - timing_per_token_ms/adv:0.000487665052443401 - perf/total_num_tokens:326140 - perf/time_per_step:48.80308444797993 - perf/throughput:835.3467913171513 (TaskRunner pid=2049544) Training Progress: 97%|█████████▋| 97/100 [1:17:40<02:27, 49.03s/it] (TaskRunner pid=2049544) step:98 - global_seqlen/min:23082 - global_seqlen/max:43864 - global_seqlen/minmax_diff:20782 - global_seqlen/balanced_min:32541 - global_seqlen/balanced_max:33196 - global_seqlen/mean:32691.375 - actor/entropy:0.3132493495941162 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.6835976243019104 - training/rollout_probs_diff_mean:0.0034196285996586084 - training/rollout_probs_diff_std:0.009468399919569492 - training/rollout_actor_probs_pearson_corr:0.9993211627006531 - rollout_corr/rollout_is_mean:1.0001163482666016 - rollout_corr/rollout_is_ratio_fraction_high:1.5585144865326583e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.5585144865326583e-05 - rollout_corr/rollout_is_max:5.03674840927124 - rollout_corr/rollout_is_min:0.0648747980594635 - rollout_corr/rollout_is_std:0.03038937970995903 - rollout_corr/rollout_is_eff_sample_size:0.9990775757045947 - rollout_corr/rollout_is_seq_mean:1.0001397132873535 - rollout_corr/rollout_is_seq_std:0.001354765729047358 - rollout_corr/rollout_is_seq_max:1.0069159269332886 - rollout_corr/rollout_is_seq_min:0.9959137439727783 - rollout_corr/rollout_is_seq_max_deviation:0.006915926933288574 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2990983063355088) - rollout_corr/training_log_ppl:np.float64(0.24720221558527555) - rollout_corr/kl:np.float64(0.0005138034307572426) - rollout_corr/k3_kl:np.float64(0.0005089092589116717) - rollout_corr/rollout_ppl:np.float64(1.298400120344013) - rollout_corr/rollout_log_ppl:np.float64(0.24668841203674674) - rollout_corr/log_ppl_diff:np.float64(0.0005138035485288128) - rollout_corr/log_ppl_abs_diff:np.float64(0.0011324260121909901) - rollout_corr/log_ppl_diff_max:np.float64(0.00813785195350647) - rollout_corr/log_ppl_diff_min:np.float64(-0.005736410617828369) - rollout_corr/ppl_ratio:np.float64(1.0005151100922376) - rollout_corr/chi2_token:np.float64(0.0009681761730462313) - rollout_corr/chi2_seq:np.float64(4.5980356687214226) - actor/pg_loss:np.float64(-9.167857933789492e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0002514609163881687) - actor/ppo_kl:np.float64(0.00017433107315412144) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.1871425360441208) - perf/mfu/actor:np.float64(0.09212177950831188) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.56390380859375) - actor/lr:np.float64(1e-06) - training/global_step:98 - training/epoch:4 - critic/score/mean:0.70703125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.70703125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:0.0003707965661305934 - critic/advantages/max:0.875 - critic/advantages/min:-0.75 - critic/returns/mean:0.0003707965661305934 - critic/returns/max:0.875 - critic/returns/min:-0.75 - response_length/mean:751.91796875 - response_length/max:4148.0 - response_length/min:99.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:751.91796875 - response_length_non_aborted/max:4148.0 - response_length_non_aborted/min:99.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.6875 - prompt_length/max:350.0 - prompt_length/min:208.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00010955892503261566 - timing_s/agent_loop/generate_sequences/min:np.float64(1.1268691644072533) - timing_s/agent_loop/generate_sequences/max:np.float64(24.762351796031) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.324899325562001) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(24.762351796031) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:304 - timing_s/agent_loop/slowest/response_length:4148 - timing_s/gen:30.538867769762874 - timing_s/reward:0.07450433447957039 - timing_s/old_log_prob:2.632541459053755 - timing_s/adv:0.1884304340928793 - timing_s/update_actor:10.359910435974598 - timing_s/step:43.87974337860942 - timing_s/stop_profile:0.000130457803606987 - timing_per_token_ms/update_actor:0.03961255237801484 - timing_per_token_ms/gen:0.15865088637787156 - timing_per_token_ms/adv:0.0007204898619776597 - perf/total_num_tokens:261531 - perf/time_per_step:43.87974337860942 - perf/throughput:745.0220188830104 (TaskRunner pid=2049544) Training Progress: 98%|█████████▊| 98/100 [1:18:24<01:34, 47.50s/it] (TaskRunner pid=2049544) step:99 - global_seqlen/min:21119 - global_seqlen/max:73608 - global_seqlen/minmax_diff:52489 - global_seqlen/balanced_min:37066 - global_seqlen/balanced_max:37818 - global_seqlen/mean:37233.375 - actor/entropy:0.36280471086502075 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.29852432012557983 - training/rollout_probs_diff_mean:0.0034003094770014286 - training/rollout_probs_diff_std:0.009052536450326443 - training/rollout_actor_probs_pearson_corr:0.9994275569915771 - rollout_corr/rollout_is_mean:0.9999655485153198 - rollout_corr/rollout_is_ratio_fraction_high:4.338112375990022e-06 - rollout_corr/rollout_is_ratio_fraction_low:1.7352449503960088e-05 - rollout_corr/rollout_is_max:2.0474398136138916 - rollout_corr/rollout_is_min:0.40364018082618713 - rollout_corr/rollout_is_std:0.029825152829289436 - rollout_corr/rollout_is_eff_sample_size:0.9991112508590329 - rollout_corr/rollout_is_seq_mean:0.9999865293502808 - rollout_corr/rollout_is_seq_std:0.0013986814301460981 - rollout_corr/rollout_is_seq_max:1.0060322284698486 - rollout_corr/rollout_is_seq_min:0.9942169785499573 - rollout_corr/rollout_is_seq_max_deviation:0.006032228469848633 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.2896207650192082) - rollout_corr/training_log_ppl:np.float64(0.2305254971361137) - rollout_corr/kl:np.float64(0.0005066159342224097) - rollout_corr/k3_kl:np.float64(0.0004654821964606981) - rollout_corr/rollout_ppl:np.float64(1.2889360883273184) - rollout_corr/rollout_log_ppl:np.float64(0.23001887879217975) - rollout_corr/log_ppl_diff:np.float64(0.0005066183439339511) - rollout_corr/log_ppl_abs_diff:np.float64(0.0010047785108326934) - rollout_corr/log_ppl_diff_max:np.float64(0.0074303895235061646) - rollout_corr/log_ppl_diff_min:np.float64(-0.004489123821258545) - rollout_corr/ppl_ratio:np.float64(1.0005076131783426) - rollout_corr/chi2_token:np.float64(0.000825623283162713) - rollout_corr/chi2_seq:np.float64(0.5253964632283896) - actor/pg_loss:np.float64(7.684435695409775e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.00010054797257907921) - actor/ppo_kl:np.float64(4.189955548028479e-05) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.08311776397749782) - perf/mfu/actor:np.float64(0.10069730946959708) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.47393417358398) - actor/lr:np.float64(1e-06) - training/global_step:99 - training/epoch:4 - critic/score/mean:0.86328125 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.86328125 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.017708174884319305 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.017708174884319305 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:900.44921875 - response_length/max:6057.0 - response_length/min:82.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:900.44921875 - response_length_non_aborted/max:6057.0 - response_length_non_aborted/min:82.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:263.09375 - prompt_length/max:365.0 - prompt_length/min:157.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:0.00011433474719524384 - timing_s/agent_loop/generate_sequences/min:np.float64(0.8707099817693233) - timing_s/agent_loop/generate_sequences/max:np.float64(35.631088424474) - timing_s/agent_loop/generate_sequences/mean:np.float64(7.754026362330478) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(35.631088424474) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:299 - timing_s/agent_loop/slowest/response_length:6057 - timing_s/gen:41.34765339829028 - timing_s/reward:0.08056613802909851 - timing_s/old_log_prob:2.6803389824926853 - timing_s/adv:0.13033771142363548 - timing_s/update_actor:10.983067074790597 - timing_s/step:55.307873111218214 - timing_s/stop_profile:0.00011896714568138123 - timing_per_token_ms/update_actor:0.03687238624886475 - timing_per_token_ms/gen:0.17937077152588887 - timing_per_token_ms/adv:0.000437570161930108 - perf/total_num_tokens:297867 - perf/time_per_step:55.307873111218214 - perf/throughput:673.2020760430919 (TaskRunner pid=2049544) Training Progress: 99%|█████████▉| 99/100 [1:19:20<00:49, 49.85s/it] (TaskRunner pid=2049544) test_gen_batch meta info: {'eos_token_id': 151645, 'pad_token_id': 151643, 'recompute_log_prob': False, 'do_sample': True, 'validate': True, 'global_steps': 100} (TaskRunner pid=2049544) validation generation end (TaskRunner pid=2049544) (TaskRunner pid=2049544) [prompt] system (TaskRunner pid=2049544) (TaskRunner pid=2049544) Given a question and four options, please select the right answer. Respond in the following format: (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) ... (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) For the answer, only output the letter corresponding to the correct option (A, B, C, or D), and nothing else. Do not restate the answer text. For example, if the answer is "A", just output: (TaskRunner pid=2049544) (TaskRunner pid=2049544) A (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) user (TaskRunner pid=2049544) A charged particle produces an electric field with a magnitude of $2.0 \mathrm{~N} / \mathrm{C}$ at a point that is $50 \mathrm{~cm}$ away from the particle. What is the magnitude of the particle's charge? Recall that the electric field $E$ due to a point charge $q$ at a distance $r$ is given by $E = k\frac{q}{r^2}$, where $k = 8.99 \times 10^9 \mathrm{N} \mathrm{m}^2 / \mathrm{C}^2$. (TaskRunner pid=2049544) (TaskRunner pid=2049544) A: 50 pC (TaskRunner pid=2049544) B: 56 pC (TaskRunner pid=2049544) C: 60 pC (TaskRunner pid=2049544) D: 64 pC (TaskRunner pid=2049544) Please reason step by step. (TaskRunner pid=2049544) assistant (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) [response] (TaskRunner pid=2049544) We are given: (TaskRunner pid=2049544) (TaskRunner pid=2049544) - Electric field $ E = 2.0 \, \mathrm{N/C} $ (TaskRunner pid=2049544) - Distance $ r = 50 \, \mathrm{cm} = 0.50 \, \mathrm{m} $ (TaskRunner pid=2049544) - Coulomb's constant $ k = 8.99 \times 10^9 \, \mathrm{N \cdot m^2/C^2} $ (TaskRunner pid=2049544) (TaskRunner pid=2049544) We use the formula for the electric field due to a point charge: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) E = k \frac{q}{r^2} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Solving for $ q $: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{E r^2}{k} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Substitute the values: (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{(2.0)(0.50)^2}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{2.0 \times 0.25}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q = \frac{0.50}{8.99 \times 10^9} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx \frac{0.50}{8.99 \times 10^9} \approx 5.56 \times 10^{-11} \, \mathrm{C} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) Convert to picocoulombs (1 pC = $ 10^{-12} $ C): (TaskRunner pid=2049544) (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) q \approx 5.56 \times 10^{-11} \, \mathrm{C} = 55.6 \, \mathrm{pC} (TaskRunner pid=2049544) $$ (TaskRunner pid=2049544) (TaskRunner pid=2049544) The closest answer is **B: 56 pC**. (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) (TaskRunner pid=2049544) B (TaskRunner pid=2049544) (TaskRunner pid=2049544) [ground_truth] B (TaskRunner pid=2049544) [score] 1.0 (TaskRunner pid=2049544) [acc] 1.0 (TaskRunner pid=2049544) [pred] B (TaskRunner pid=2049544) [incorrect_format] 1 (TaskRunner pid=2049544) [feedback] (TaskRunner pid=2049544) local_global_step_folder: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_100 (WorkerDict pid=2049932) Checkpoint manager remove previous save local path: /mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8/global_step_90/actor (TaskRunner pid=2049544) step:100 - global_seqlen/min:17010 - global_seqlen/max:43433 - global_seqlen/minmax_diff:26423 - global_seqlen/balanced_min:31299 - global_seqlen/balanced_max:31491 - global_seqlen/mean:31419.0 - actor/entropy:0.37953582406044006 - perf/mfu/actor_infer:0 - training/rollout_probs_diff_valid:1 - training/rollout_probs_diff_max:0.630595862865448 - training/rollout_probs_diff_mean:0.0036211644764989614 - training/rollout_probs_diff_std:0.00978316180408001 - training/rollout_actor_probs_pearson_corr:0.999374508857727 - rollout_corr/rollout_is_mean:1.0000176429748535 - rollout_corr/rollout_is_ratio_fraction_high:1.0963950444420334e-05 - rollout_corr/rollout_is_ratio_fraction_low:1.644592521188315e-05 - rollout_corr/rollout_is_max:2.3022942543029785 - rollout_corr/rollout_is_min:0.3131349980831146 - rollout_corr/rollout_is_std:0.031683772802352905 - rollout_corr/rollout_is_eff_sample_size:0.998997264270168 - rollout_corr/rollout_is_seq_mean:1.0000271797180176 - rollout_corr/rollout_is_seq_std:0.0016033316496759653 - rollout_corr/rollout_is_seq_max:1.0072500705718994 - rollout_corr/rollout_is_seq_min:0.9959959983825684 - rollout_corr/rollout_is_seq_max_deviation:0.007250070571899414 - rollout_corr/rollout_is_seq_fraction_high:0.0 - rollout_corr/rollout_is_seq_fraction_low:0.0 - rollout_corr/training_ppl:np.float64(1.3458213401027024) - rollout_corr/training_log_ppl:np.float64(0.26966726542741526) - rollout_corr/kl:np.float64(0.00037315004849824973) - rollout_corr/k3_kl:np.float64(0.0005444729254122649) - rollout_corr/rollout_ppl:np.float64(1.3452776027843356) - rollout_corr/rollout_log_ppl:np.float64(0.2692941162531497) - rollout_corr/log_ppl_diff:np.float64(0.0003731491742655635) - rollout_corr/log_ppl_abs_diff:np.float64(0.0012809229083359241) - rollout_corr/log_ppl_diff_max:np.float64(0.0049625784158706665) - rollout_corr/log_ppl_diff_min:np.float64(-0.0106421560049057) - rollout_corr/ppl_ratio:np.float64(1.0003747479058802) - rollout_corr/chi2_token:np.float64(0.0014326744712889194) - rollout_corr/chi2_seq:np.float64(1.9170411999803036) - actor/pg_loss:np.float64(-7.347844075411558e-05) - actor/kl_loss:np.float64(0.0) - actor/pg_clipfrac:np.float64(0.0001629320678375734) - actor/ppo_kl:np.float64(-0.00012778303023264215) - actor/pg_clipfrac_lower:np.float64(0.0) - actor/grad_norm:np.float64(0.12031856318935752) - perf/mfu/actor:np.float64(0.0881824181802986) - perf/max_memory_allocated_gb:np.float64(123.32309532165527) - perf/max_memory_reserved_gb:np.float64(129.08984375) - perf/cpu_memory_used_gb:np.float64(107.57986450195312) - actor/lr:np.float64(1e-06) - val-aux/sciknoweval/reward/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/reward/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/reward/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/reward/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/reward/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/reward/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/reward/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/reward/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/reward/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/reward/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/reward/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/reward/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/reward/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/reward/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/reward/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/reward/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/reward/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/reward/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/reward/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/reward/maj@8/std:np.float64(0.08949529688241312) - val-aux/sciknoweval/reward/best@16/mean:np.float64(0.8910875) - val-aux/sciknoweval/reward/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/reward/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/reward/worst@16/std:np.float64(0.054003469751280585) - val-aux/sciknoweval/reward/maj@16/mean:np.float64(0.7157250000000001) - val-aux/sciknoweval/reward/maj@16/std:np.float64(0.05584489583302406) - val-aux/sciknoweval/score/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/score/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/score/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/score/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/score/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/score/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/score/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/score/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/score/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/score/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/score/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/score/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/score/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/score/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/score/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/score/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/score/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/score/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/score/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/score/maj@8/std:np.float64(0.08949529688241312) - val-aux/sciknoweval/score/best@16/mean:np.float64(0.8910875) - val-aux/sciknoweval/score/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/score/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/score/worst@16/std:np.float64(0.054003469751280585) - val-aux/sciknoweval/score/maj@16/mean:np.float64(0.7157250000000001) - val-aux/sciknoweval/score/maj@16/std:np.float64(0.05584489583302406) - val-core/sciknoweval/acc/mean@16:np.float64(0.6828125) - val-aux/sciknoweval/acc/std@16:np.float64(0.17979486326514935) - val-aux/sciknoweval/acc/best@2/mean:np.float64(0.7532) - val-aux/sciknoweval/acc/best@2/std:np.float64(0.1476854115376502) - val-aux/sciknoweval/acc/worst@2/mean:np.float64(0.6133875) - val-aux/sciknoweval/acc/worst@2/std:np.float64(0.15413287282151064) - val-aux/sciknoweval/acc/maj@2/mean:np.float64(0.6832499999999999) - val-aux/sciknoweval/acc/maj@2/std:np.float64(0.17928347494256552) - val-aux/sciknoweval/acc/best@4/mean:np.float64(0.807675) - val-aux/sciknoweval/acc/best@4/std:np.float64(0.11309093981217563) - val-aux/sciknoweval/acc/worst@4/mean:np.float64(0.5515125000000001) - val-aux/sciknoweval/acc/worst@4/std:np.float64(0.12241577016075818) - val-aux/sciknoweval/acc/maj@4/mean:np.float64(0.7018125) - val-aux/sciknoweval/acc/maj@4/std:np.float64(0.13581544160753714) - val-aux/sciknoweval/acc/best@8/mean:np.float64(0.8526375000000002) - val-aux/sciknoweval/acc/best@8/std:np.float64(0.0888708326383256) - val-aux/sciknoweval/acc/worst@8/mean:np.float64(0.4995375) - val-aux/sciknoweval/acc/worst@8/std:np.float64(0.08946281937139845) - val-aux/sciknoweval/acc/maj@8/mean:np.float64(0.710175) - val-aux/sciknoweval/acc/maj@8/std:np.float64(0.08949529688241312) - val-core/sciknoweval/acc/best@16/mean:np.float64(0.8910875) - val-core/sciknoweval/acc/best@16/std:np.float64(0.061136881006381635) - val-aux/sciknoweval/acc/worst@16/mean:np.float64(0.46395) - val-aux/sciknoweval/acc/worst@16/std:np.float64(0.054003469751280585) - val-core/sciknoweval/acc/maj@16/mean:np.float64(0.7157250000000001) - val-core/sciknoweval/acc/maj@16/std:np.float64(0.05584489583302406) - val-aux/sciknoweval/incorrect_format/mean@16:np.float64(0.9859375) - val-aux/sciknoweval/incorrect_format/std@16:np.float64(0.03383512807052093) - val-aux/sciknoweval/incorrect_format/best@2/mean:np.float64(0.99745) - val-aux/sciknoweval/incorrect_format/best@2/std:np.float64(0.014287439995541685) - val-aux/sciknoweval/incorrect_format/worst@2/mean:np.float64(0.9741250000000001) - val-aux/sciknoweval/incorrect_format/worst@2/std:np.float64(0.042452979144002284) - val-aux/sciknoweval/incorrect_format/maj@2/mean:np.float64(0.9857875) - val-aux/sciknoweval/incorrect_format/maj@2/std:np.float64(0.03387729307844336) - val-aux/sciknoweval/incorrect_format/best@4/mean:np.float64(0.9998374999999999) - val-aux/sciknoweval/incorrect_format/best@4/std:np.float64(0.002749370152057837) - val-aux/sciknoweval/incorrect_format/worst@4/mean:np.float64(0.9560875) - val-aux/sciknoweval/incorrect_format/worst@4/std:np.float64(0.04764508939236297) - val-aux/sciknoweval/incorrect_format/maj@4/mean:np.float64(0.9946999999999999) - val-aux/sciknoweval/incorrect_format/maj@4/std:np.float64(0.021483511676811038) - val-aux/sciknoweval/incorrect_format/best@8/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@8/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@8/mean:np.float64(0.933025) - val-aux/sciknoweval/incorrect_format/worst@8/std:np.float64(0.0436722700672832) - val-aux/sciknoweval/incorrect_format/maj@8/mean:np.float64(0.9988624999999999) - val-aux/sciknoweval/incorrect_format/maj@8/std:np.float64(0.009237722502874987) - val-aux/sciknoweval/incorrect_format/best@16/mean:np.float64(1.0) - val-aux/sciknoweval/incorrect_format/best@16/std:np.float64(0.0) - val-aux/sciknoweval/incorrect_format/worst@16/mean:np.float64(0.9139250000000001) - val-aux/sciknoweval/incorrect_format/worst@16/std:np.float64(0.029568343249240757) - val-aux/sciknoweval/incorrect_format/maj@16/mean:np.float64(0.9998750000000001) - val-aux/sciknoweval/incorrect_format/maj@16/std:np.float64(0.0017543234207183663) - val-aux/num_turns/min:np.int32(2) - val-aux/num_turns/max:np.int32(2) - val-aux/num_turns/mean:np.float64(2.0) - training/global_step:100 - training/epoch:4 - critic/score/mean:0.7265625 - critic/score/max:1.0 - critic/score/min:0.0 - critic/rewards/mean:0.7265625 - critic/rewards/max:1.0 - critic/rewards/min:0.0 - critic/advantages/mean:-0.0018172748386859894 - critic/advantages/max:0.875 - critic/advantages/min:-0.875 - critic/returns/mean:-0.0018172748386859894 - critic/returns/max:0.875 - critic/returns/min:-0.875 - response_length/mean:712.5625 - response_length/max:3652.0 - response_length/min:110.0 - response_length/clip_ratio:0.0 - response_length_non_aborted/mean:712.5625 - response_length_non_aborted/max:3652.0 - response_length_non_aborted/min:110.0 - response_length_non_aborted/clip_ratio:0.0 - response/aborted_ratio:0.0 - prompt_length/mean:269.28125 - prompt_length/max:383.0 - prompt_length/min:191.0 - prompt_length/clip_ratio:0.0 - num_turns/min:np.int32(2) - num_turns/max:np.int32(2) - num_turns/mean:np.float64(2.0) - timing_s/start_profile:8.623674511909485e-05 - timing_s/agent_loop/generate_sequences/min:np.float64(1.2800081353634596) - timing_s/agent_loop/generate_sequences/max:np.float64(22.433935675770044) - timing_s/agent_loop/generate_sequences/mean:np.float64(6.539074072250514) - timing_s/agent_loop/tool_calls/min:np.float64(0.0) - timing_s/agent_loop/tool_calls/max:np.float64(0.0) - timing_s/agent_loop/tool_calls/mean:np.float64(0.0) - timing_s/agent_loop/slowest/generate_sequences:np.float64(22.433935675770044) - timing_s/agent_loop/slowest/tool_calls:np.float64(0.0) - timing_s/agent_loop/slowest/prompt_length:342 - timing_s/agent_loop/slowest/response_length:3652 - timing_s/gen:28.36895477026701 - timing_s/reward:0.07503514550626278 - timing_s/old_log_prob:2.573247427120805 - timing_s/adv:0.12070851400494576 - timing_s/update_actor:10.477075356990099 - timing_s/step:41.70409296452999 - timing_s/testing:119.00530271604657 - timing_s/save_checkpoint:6.902973407879472 - timing_s/stop_profile:0.00019830837845802307 - timing_per_token_ms/update_actor:0.041682880410699334 - timing_per_token_ms/gen:0.15551790835380125 - timing_per_token_ms/adv:0.00048023693467704956 - perf/total_num_tokens:251352 - perf/time_per_step:41.70409296452999 - perf/throughput:753.3792912537954 (TaskRunner pid=2049544) ("Final validation metrics: {'val-aux/sciknoweval/reward/mean@16': " (TaskRunner pid=2049544) "np.float64(0.6828125), 'val-aux/sciknoweval/reward/std@16': " (TaskRunner pid=2049544) "np.float64(0.17979486326514935), 'val-aux/sciknoweval/reward/best@2/mean': " (TaskRunner pid=2049544) "np.float64(0.7532), 'val-aux/sciknoweval/reward/best@2/std': " (TaskRunner pid=2049544) "np.float64(0.1476854115376502), 'val-aux/sciknoweval/reward/worst@2/mean': " (TaskRunner pid=2049544) "np.float64(0.6133875), 'val-aux/sciknoweval/reward/worst@2/std': " (TaskRunner pid=2049544) "np.float64(0.15413287282151064), 'val-aux/sciknoweval/reward/maj@2/mean': " (TaskRunner pid=2049544) "np.float64(0.6832499999999999), 'val-aux/sciknoweval/reward/maj@2/std': " (TaskRunner pid=2049544) "np.float64(0.17928347494256552), 'val-aux/sciknoweval/reward/best@4/mean': " (TaskRunner pid=2049544) "np.float64(0.807675), 'val-aux/sciknoweval/reward/best@4/std': " (TaskRunner pid=2049544) "np.float64(0.11309093981217563), 'val-aux/sciknoweval/reward/worst@4/mean': " (TaskRunner pid=2049544) "np.float64(0.5515125000000001), 'val-aux/sciknoweval/reward/worst@4/std': " (TaskRunner pid=2049544) "np.float64(0.12241577016075818), 'val-aux/sciknoweval/reward/maj@4/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) Training Progress: 100%|██████████| 100/100 [1:22:07<00:00, 85.21s/it] (TaskRunner pid=2049544) "np.float64(0.7018125), 'val-aux/sciknoweval/reward/maj@4/std': " (TaskRunner pid=2049544) "np.float64(0.13581544160753714), 'val-aux/sciknoweval/reward/best@8/mean': " (TaskRunner pid=2049544) "np.float64(0.8526375000000002), 'val-aux/sciknoweval/reward/best@8/std': " (TaskRunner pid=2049544) "np.float64(0.0888708326383256), 'val-aux/sciknoweval/reward/worst@8/mean': " (TaskRunner pid=2049544) "np.float64(0.4995375), 'val-aux/sciknoweval/reward/worst@8/std': " (TaskRunner pid=2049544) "np.float64(0.08946281937139845), 'val-aux/sciknoweval/reward/maj@8/mean': " (TaskRunner pid=2049544) "np.float64(0.710175), 'val-aux/sciknoweval/reward/maj@8/std': " (TaskRunner pid=2049544) "np.float64(0.08949529688241312), 'val-aux/sciknoweval/reward/best@16/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "np.float64(0.8910875), 'val-aux/sciknoweval/reward/best@16/std': " (TaskRunner pid=2049544) 'np.float64(0.061136881006381635), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/reward/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/mean@16': np.float64(0.6828125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/std@16': np.float64(0.17979486326514935), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@2/mean': np.float64(0.7532), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@2/std': np.float64(0.1476854115376502), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@2/mean': np.float64(0.6133875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@2/std': np.float64(0.15413287282151064), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@2/mean': np.float64(0.6832499999999999), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@2/std': np.float64(0.17928347494256552), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@4/mean': np.float64(0.807675), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@4/std': np.float64(0.11309093981217563), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@4/mean': np.float64(0.5515125000000001), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@4/std': np.float64(0.12241577016075818), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@4/mean': np.float64(0.7018125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@4/std': np.float64(0.13581544160753714), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@8/mean': np.float64(0.8526375000000002), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@8/std': np.float64(0.0888708326383256), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@8/mean': np.float64(0.4995375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@8/std': np.float64(0.08946281937139845), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@8/mean': np.float64(0.710175), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@8/std': np.float64(0.08949529688241312), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@16/mean': np.float64(0.8910875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/best@16/std': np.float64(0.061136881006381635), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/score/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/mean@16': np.float64(0.6828125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/std@16': np.float64(0.17979486326514935), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@2/mean': np.float64(0.7532), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@2/std': np.float64(0.1476854115376502), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@2/mean': np.float64(0.6133875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@2/std': np.float64(0.15413287282151064), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@2/mean': np.float64(0.6832499999999999), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@2/std': np.float64(0.17928347494256552), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@4/mean': np.float64(0.807675), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@4/std': np.float64(0.11309093981217563), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@4/mean': np.float64(0.5515125000000001), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@4/std': np.float64(0.12241577016075818), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@4/mean': np.float64(0.7018125), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@4/std': np.float64(0.13581544160753714), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@8/mean': np.float64(0.8526375000000002), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/best@8/std': np.float64(0.0888708326383256), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@8/mean': np.float64(0.4995375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@8/std': np.float64(0.08946281937139845), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@8/mean': np.float64(0.710175), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/maj@8/std': np.float64(0.08949529688241312), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/best@16/mean': np.float64(0.8910875), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/best@16/std': np.float64(0.061136881006381635), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@16/mean': np.float64(0.46395), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/acc/worst@16/std': np.float64(0.054003469751280585), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/maj@16/mean': np.float64(0.7157250000000001), " (TaskRunner pid=2049544) "'val-core/sciknoweval/acc/maj@16/std': np.float64(0.05584489583302406), " (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/mean@16': np.float64(0.9859375), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/std@16': " (TaskRunner pid=2049544) 'np.float64(0.03383512807052093), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@2/mean': np.float64(0.99745), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@2/std': " (TaskRunner pid=2049544) 'np.float64(0.014287439995541685), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@2/mean': " (TaskRunner pid=2049544) 'np.float64(0.9741250000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@2/std': " (TaskRunner pid=2049544) 'np.float64(0.042452979144002284), ' (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@2/mean': np.float64(0.9857875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@2/std': " (TaskRunner pid=2049544) 'np.float64(0.03387729307844336), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@4/mean': " (TaskRunner pid=2049544) 'np.float64(0.9998374999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@4/std': " (TaskRunner pid=2049544) 'np.float64(0.002749370152057837), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@4/mean': np.float64(0.9560875), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@4/std': " (TaskRunner pid=2049544) 'np.float64(0.04764508939236297), ' (TaskRunner pid=2049544) (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@4/mean': " (TaskRunner pid=2049544) 'np.float64(0.9946999999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@4/std': " (TaskRunner pid=2049544) 'np.float64(0.021483511676811038), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@8/mean': np.float64(1.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@8/std': np.float64(0.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@8/mean': np.float64(0.933025), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@8/std': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) 'np.float64(0.0436722700672832), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@8/mean': " (TaskRunner pid=2049544) 'np.float64(0.9988624999999999), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@8/std': " (TaskRunner pid=2049544) 'np.float64(0.009237722502874987), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@16/mean': np.float64(1.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/best@16/std': np.float64(0.0), " (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@16/mean': " (TaskRunner pid=2049544) (TaskRunner pid=2049544) 'np.float64(0.9139250000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/worst@16/std': " (TaskRunner pid=2049544) 'np.float64(0.029568343249240757), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@16/mean': " (TaskRunner pid=2049544) 'np.float64(0.9998750000000001), ' (TaskRunner pid=2049544) "'val-aux/sciknoweval/incorrect_format/maj@16/std': " (TaskRunner pid=2049544) "np.float64(0.0017543234207183663), 'val-aux/num_turns/min': np.int32(2), " (TaskRunner pid=2049544) "'val-aux/num_turns/max': np.int32(2), 'val-aux/num_turns/mean': " (TaskRunner pid=2049544) 'np.float64(2.0)}') (TaskRunner pid=2049544) Training Progress: 100%|██████████| 100/100 [1:22:07<00:00, 49.28s/it] (TaskRunner pid=2049544) (TaskRunner pid=2049544) wandb: updating run metadata (TaskRunner pid=2049544) wandb: uploading output.log; uploading wandb-summary.json (TaskRunner pid=2049544) wandb: uploading output.log; uploading config.yaml (TaskRunner pid=2049544) wandb: uploading history steps 98-99, summary, console lines 919-1044 (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: Run history: (TaskRunner pid=2049544) wandb: actor/entropy ▁▁▂▁▂▁▁▁▂▃▃▂▂▂▃▂▃▃▃▃▄▄▃▄▅▄▆▄▇▆▅▄▆▆▆▅▇█▇▇ (TaskRunner pid=2049544) wandb: actor/grad_norm █▄▇▃█▆▃▇▆▁▄▄▃▃▃▂▁▂▁▂▄▃▂▂▂▂▄▂▃▃▂▃▂▃▂▁▃▂▂▁ (TaskRunner pid=2049544) wandb: actor/kl_loss ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ (TaskRunner pid=2049544) wandb: actor/lr ▁▂▃▄▆███████████████████████████████████ (TaskRunner pid=2049544) wandb: actor/pg_clipfrac ▂▄▅▃▆▇▄▃█▃▃▄▂▂▃▃▂▂▂▂▃▃▃▂▂▁▁▁▁▂▁▁▁▂▂▂▁▁▂▁ (TaskRunner pid=2049544) wandb: actor/pg_clipfrac_lower ▁▁▁▁▁▁▆▁▁▄▁▃▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▁ (TaskRunner pid=2049544) wandb: actor/pg_loss ▆▂▅▅█▆▅▃▃▅▅▄▃▄▅▄▄▃▂▅▁▂▄▄▁▄▆▃▅▄▅▅▂▅▅▄▅▃▅▃ (TaskRunner pid=2049544) wandb: actor/ppo_kl ▁▂▂▃▄▅▃▁▄▃▄▂▄▁▂▂▃▃▂▂▇▄▂▃▂▂▄▄▃▁▄▃▁▂█▃▃▂▃▂ (TaskRunner pid=2049544) wandb: critic/advantages/max █████████▅█████████████▅███▁████████████ (TaskRunner pid=2049544) wandb: critic/advantages/mean ▅▇▆▅▂▄▃▅▅▅▃▂▃▆▃▇▆▂█▂▄▅▃▃▅▁▅▄▄▄▄▅▄▆▆▅▄▃▂▅ (TaskRunner pid=2049544) wandb: +204 ... (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: Run summary: (TaskRunner pid=2049544) wandb: actor/entropy 0.37954 (TaskRunner pid=2049544) wandb: actor/grad_norm 0.12032 (TaskRunner pid=2049544) wandb: actor/kl_loss 0 (TaskRunner pid=2049544) wandb: actor/lr 0.0 (TaskRunner pid=2049544) wandb: actor/pg_clipfrac 0.00016 (TaskRunner pid=2049544) wandb: actor/pg_clipfrac_lower 0 (TaskRunner pid=2049544) wandb: actor/pg_loss -7e-05 (TaskRunner pid=2049544) wandb: actor/ppo_kl -0.00013 (TaskRunner pid=2049544) wandb: critic/advantages/max 0.875 (TaskRunner pid=2049544) wandb: critic/advantages/mean -0.00182 (TaskRunner pid=2049544) wandb: +204 ... (TaskRunner pid=2049544) wandb: (TaskRunner pid=2049544) wandb: 🚀 View run qwen3gen-physics-GRPO-Qwen-Qwen3-4B-mbs8-train32-rollout8-lr1e-6-vllm0.8 at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root/runs/o8ivivjg (TaskRunner pid=2049544) wandb: ⭐️ View project at: https://wandb.ai/seongryongjung-chung-ang-university/SDPO-root (TaskRunner pid=2049544) wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s) (TaskRunner pid=2049544) wandb: Find logs at: ./wandb/run-20260702_073403-o8ivivjg/logs main_ppo exit code: 0 [2026-07-02 08:56:17] Finished physics grpo [2026-07-02 08:56:17] Starting physics rlsd_tr Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Dataset: physics Method: rlsd_tr Config: rlsd Model: Qwen/Qwen3-4B Train: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet Val: /mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet Ray tmp: /tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B 2026-07-02 08:56:19,506 INFO scripts.py:1364 -- Did not find any active Ray processes. Experiment: qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 Config: rlsd Task: datasets/sciknoweval/physics Arguments: max_model_len=10240 data.train_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet] data.val_files=[/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet] data.train_batch_size=32 data.train_max_samples=3200 data.max_prompt_length=2048 data.max_response_length=8192 data.apply_chat_template_kwargs.enable_thinking=false trainer.group_name=QWEN3-RLSD-TR-GRPO-matched-generalization trainer.n_gpus_per_node=8 trainer.total_training_steps=100 trainer.val_before_train=False trainer.save_freq=10 trainer.test_freq=10 trainer.default_local_dir=/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 custom_reward_function.path=/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py +ray_kwargs.ray_init._temp_dir=/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B +ray_kwargs.ray_init.include_dashboard=False +ray_kwargs.ray_init.runtime_env.env_vars.USER=root +ray_kwargs.ray_init.runtime_env.env_vars.TASK=datasets/sciknoweval/physics +ray_kwargs.ray_init.runtime_env.env_vars.EXPERIMENT=qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8 actor_rollout_ref.model.path=Qwen/Qwen3-4B actor_rollout_ref.actor.optim.lr=1e-6 actor_rollout_ref.actor.optim.lr_warmup_steps=10 actor_rollout_ref.actor.optim.weight_decay=0.01 actor_rollout_ref.actor.grad_clip=1.0 actor_rollout_ref.actor.ppo_mini_batch_size=8 actor_rollout_ref.actor.clip_ratio_high=0.28 actor_rollout_ref.actor.clip_ratio_low=0.2 actor_rollout_ref.actor.ppo_max_token_len_per_gpu=10240 actor_rollout_ref.rollout.n=8 actor_rollout_ref.rollout.temperature=1.0 actor_rollout_ref.rollout.top_p=1.0 actor_rollout_ref.rollout.max_model_len=10240 actor_rollout_ref.rollout.max_num_batched_tokens=10240 actor_rollout_ref.rollout.gpu_memory_utilization=0.8 actor_rollout_ref.rollout.calculate_log_probs=True actor_rollout_ref.rollout.val_kwargs.n=16 actor_rollout_ref.rollout.val_kwargs.temperature=0.6 actor_rollout_ref.rollout.val_kwargs.top_p=0.95 actor_rollout_ref.rollout.val_kwargs.do_sample=True algorithm.norm_adv_by_std_in_grpo=False algorithm.rollout_correction.rollout_is=token algorithm.rollout_correction.rollout_is_threshold=2.0 actor_rollout_ref.model.use_fused_kernels=False actor_rollout_ref.actor.policy_loss.loss_mode=rlsd actor_rollout_ref.actor.self_distillation.token_reweight_lambda=0.5 actor_rollout_ref.actor.self_distillation.token_reweight_eps_w=0.2 actor_rollout_ref.actor.self_distillation.token_reweight_decay_steps=0 actor_rollout_ref.actor.self_distillation.teacher_regularization=trust-region actor_rollout_ref.actor.self_distillation.teacher_update_rate=0.1 actor_rollout_ref.actor.self_distillation.max_reprompt_len=10240 ray init kwargs: {'num_cpus': None, '_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B', 'include_dashboard': False, 'runtime_env': {'env_vars': {'TOKENIZERS_PARALLELISM': 'true', 'NCCL_DEBUG': 'WARN', 'VLLM_LOGGING_LEVEL': 'WARN', 'VLLM_ALLOW_RUNTIME_LORA_UPDATING': 'true', 'VLLM_ALLREDUCE_USE_SYMM_MEM': '0', 'CUDA_DEVICE_MAX_CONNECTIONS': '1', 'NCCL_CUMEM_ENABLE': '0', 'USER': 'root', 'TASK': 'datasets/sciknoweval/physics', 'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8'}, 'working_dir': None}} 2026-07-02 08:56:29,254 INFO worker.py:2007 -- Started a local Ray instance. /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/ray/_private/worker.py:2046: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0 warnings.warn( (TaskRunner pid=2074004) TaskRunner hostname: mole-workspace-rw, PID: 2074004 (TaskRunner pid=2074004) {'actor_rollout_ref': {'actor': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2074004) 'calculate_entropy': False, (TaskRunner pid=2074004) 'calculate_sum_pi_squared': False, (TaskRunner pid=2074004) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2074004) 'async_save': False, (TaskRunner pid=2074004) 'load_contents': ['model', (TaskRunner pid=2074004) 'optimizer', (TaskRunner pid=2074004) 'extra'], (TaskRunner pid=2074004) 'save_contents': ['model', (TaskRunner pid=2074004) 'optimizer', (TaskRunner pid=2074004) 'extra']}, (TaskRunner pid=2074004) 'clip_ratio': 0.2, (TaskRunner pid=2074004) 'clip_ratio_c': 3.0, (TaskRunner pid=2074004) 'clip_ratio_high': 0.28, (TaskRunner pid=2074004) 'clip_ratio_low': 0.2, (TaskRunner pid=2074004) 'data_loader_seed': 42, (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_coeff': 0, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'freeze_vision_tower': False, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': False, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'grad_clip': 1.0, (TaskRunner pid=2074004) 'kl_loss_coef': 0.001, (TaskRunner pid=2074004) 'kl_loss_type': 'low_var_kl', (TaskRunner pid=2074004) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2074004) 'loss_scale_factor': None, (TaskRunner pid=2074004) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2074004) 'betas': [0.9, 0.999], (TaskRunner pid=2074004) 'clip_grad': 1.0, (TaskRunner pid=2074004) 'lr': 1e-06, (TaskRunner pid=2074004) 'lr_scheduler_type': 'constant', (TaskRunner pid=2074004) 'lr_warmup_steps': 10, (TaskRunner pid=2074004) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2074004) 'min_lr_ratio': 0.0, (TaskRunner pid=2074004) 'num_cycles': 0.5, (TaskRunner pid=2074004) 'optimizer': 'AdamW', (TaskRunner pid=2074004) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2074004) 'override_optimizer_config': None, (TaskRunner pid=2074004) 'total_training_steps': -1, (TaskRunner pid=2074004) 'warmup_style': None, (TaskRunner pid=2074004) 'weight_decay': 0.01}, (TaskRunner pid=2074004) 'policy_loss': {'_target_': 'verl.workers.config.PolicyLossConfig', (TaskRunner pid=2074004) 'clip_cov_lb': 1.0, (TaskRunner pid=2074004) 'clip_cov_ratio': 0.0002, (TaskRunner pid=2074004) 'clip_cov_ub': 5.0, (TaskRunner pid=2074004) 'kl_cov_ratio': 0.0002, (TaskRunner pid=2074004) 'loss_mode': 'rlsd', (TaskRunner pid=2074004) 'ppo_kl_coef': 0.1}, (TaskRunner pid=2074004) 'ppo_epochs': 1, (TaskRunner pid=2074004) 'ppo_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'ppo_micro_batch_size': None, (TaskRunner pid=2074004) 'ppo_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'ppo_mini_batch_size': 8, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2074004) 'mode': 'disabled', (TaskRunner pid=2074004) 'record_file': None, (TaskRunner pid=2074004) 'replay_file': None}, (TaskRunner pid=2074004) 'self_distillation': {'_target_': 'verl.workers.config.SelfDistillationConfig', (TaskRunner pid=2074004) 'alpha': 0.5, (TaskRunner pid=2074004) 'distillation_add_tail': True, (TaskRunner pid=2074004) 'distillation_topk': 100, (TaskRunner pid=2074004) 'dont_reprompt_on_self_success': True, (TaskRunner pid=2074004) 'environment_feedback_only_without_solution': True, (TaskRunner pid=2074004) 'evolving_teacher': {'_target_': 'verl.workers.config.EvolvingTeacherConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'loss_weight': 0.0, (TaskRunner pid=2074004) 'mask': 'all'}, (TaskRunner pid=2074004) 'feedback_template': '\n' (TaskRunner pid=2074004) 'The ' (TaskRunner pid=2074004) 'following ' (TaskRunner pid=2074004) 'is ' (TaskRunner pid=2074004) 'feedback ' (TaskRunner pid=2074004) 'from ' (TaskRunner pid=2074004) 'your ' (TaskRunner pid=2074004) 'unsuccessful ' (TaskRunner pid=2074004) 'earlier ' (TaskRunner pid=2074004) 'attempt:\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) '{feedback_raw}', (TaskRunner pid=2074004) 'full_logit_distillation': True, (TaskRunner pid=2074004) 'include_environment_feedback': True, (TaskRunner pid=2074004) 'is_clip': 2, (TaskRunner pid=2074004) 'max_reprompt_len': 10240, (TaskRunner pid=2074004) 'remove_thinking_from_demonstration': False, (TaskRunner pid=2074004) 'reprompt_template': '{prompt}{solution}{feedback}\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) 'Correctly ' (TaskRunner pid=2074004) 'solve ' (TaskRunner pid=2074004) 'the ' (TaskRunner pid=2074004) 'original ' (TaskRunner pid=2074004) 'question.', (TaskRunner pid=2074004) 'reprompt_truncation': 'right', (TaskRunner pid=2074004) 'solution_template': '\n' (TaskRunner pid=2074004) 'Correct ' (TaskRunner pid=2074004) 'solution:\n' (TaskRunner pid=2074004) '\n' (TaskRunner pid=2074004) '{successful_previous_attempt}', (TaskRunner pid=2074004) 'srpo_dynamic_weighting': False, (TaskRunner pid=2074004) 'srpo_dynamic_weighting_temperature': 1.0, (TaskRunner pid=2074004) 'success_reward_threshold': 0.5, (TaskRunner pid=2074004) 'teacher_regularization': 'trust-region', (TaskRunner pid=2074004) 'teacher_update_rate': 0.1, (TaskRunner pid=2074004) 'token_reweight_decay_steps': 0, (TaskRunner pid=2074004) 'token_reweight_eps_w': 0.2, (TaskRunner pid=2074004) 'token_reweight_lambda': 0.5}, (TaskRunner pid=2074004) 'shuffle': False, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'sum_pi_squared_checkpointing': False, (TaskRunner pid=2074004) 'tau_neg': 1.05, (TaskRunner pid=2074004) 'tau_pos': 1.0, (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_kl_loss': False, (TaskRunner pid=2074004) 'use_prefix_grouper': False, (TaskRunner pid=2074004) 'use_remove_padding': True, (TaskRunner pid=2074004) 'use_torch_compile': True}, (TaskRunner pid=2074004) 'hybrid_engine': True, (TaskRunner pid=2074004) 'model': {'_target_': 'verl.workers.config.HFModelConfig', (TaskRunner pid=2074004) 'custom_chat_template': None, (TaskRunner pid=2074004) 'enable_activation_offload': False, (TaskRunner pid=2074004) 'enable_gradient_checkpointing': True, (TaskRunner pid=2074004) 'exclude_modules': None, (TaskRunner pid=2074004) 'external_lib': None, (TaskRunner pid=2074004) 'fused_kernel_options': {'impl_backend': 'torch'}, (TaskRunner pid=2074004) 'hf_config_path': None, (TaskRunner pid=2074004) 'lora_adapter_path': None, (TaskRunner pid=2074004) 'lora_alpha': 16, (TaskRunner pid=2074004) 'lora_rank': 0, (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'target_modules': 'all-linear', (TaskRunner pid=2074004) 'tiled_mlp': {'enabled': False, (TaskRunner pid=2074004) 'num_shards': 4}, (TaskRunner pid=2074004) 'tokenizer_path': None, (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_liger': False, (TaskRunner pid=2074004) 'use_remove_padding': True, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'nccl_timeout': 600, (TaskRunner pid=2074004) 'ref': {'_target_': 'verl.workers.config.FSDPActorConfig', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': True, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'log_prob_micro_batch_size': None, (TaskRunner pid=2074004) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'router_replay': {'_target_': 'verl.workers.config.RouterReplayConfig', (TaskRunner pid=2074004) 'mode': 'disabled', (TaskRunner pid=2074004) 'record_file': None, (TaskRunner pid=2074004) 'replay_file': None}, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_torch_compile': True}, (TaskRunner pid=2074004) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2074004) 'agent': {'_target_': 'verl.workers.config.AgentLoopConfig', (TaskRunner pid=2074004) 'agent_loop_config_path': None, (TaskRunner pid=2074004) 'custom_async_server': {'_target_': 'verl.workers.config.CustomAsyncServerConfig', (TaskRunner pid=2074004) 'name': None, (TaskRunner pid=2074004) 'path': None}, (TaskRunner pid=2074004) 'default_agent_loop': 'single_turn_agent', (TaskRunner pid=2074004) 'num_workers': 8}, (TaskRunner pid=2074004) 'calculate_log_probs': True, (TaskRunner pid=2074004) 'cudagraph_capture_sizes': None, (TaskRunner pid=2074004) 'data_parallel_size': 1, (TaskRunner pid=2074004) 'disable_log_stats': True, (TaskRunner pid=2074004) 'do_sample': True, (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'enable_chunked_prefill': True, (TaskRunner pid=2074004) 'enable_prefix_caching': True, (TaskRunner pid=2074004) 'enable_rollout_routing_replay': False, (TaskRunner pid=2074004) 'enforce_eager': False, (TaskRunner pid=2074004) 'engine_kwargs': {'sglang': {}, 'vllm': {}}, (TaskRunner pid=2074004) 'expert_parallel_size': 1, (TaskRunner pid=2074004) 'free_cache_engine': True, (TaskRunner pid=2074004) 'gpu_memory_utilization': 0.8, (TaskRunner pid=2074004) 'ignore_eos': False, (TaskRunner pid=2074004) 'layered_summon': False, (TaskRunner pid=2074004) 'load_format': 'dummy', (TaskRunner pid=2074004) 'log_prob_max_token_len_per_gpu': 10240, (TaskRunner pid=2074004) 'log_prob_micro_batch_size': None, (TaskRunner pid=2074004) 'log_prob_micro_batch_size_per_gpu': 1, (TaskRunner pid=2074004) 'log_prob_use_dynamic_bsz': False, (TaskRunner pid=2074004) 'logprobs_mode': 'processed_logprobs', (TaskRunner pid=2074004) 'max_model_len': 10240, (TaskRunner pid=2074004) 'max_num_batched_tokens': 10240, (TaskRunner pid=2074004) 'max_num_seqs': 1024, (TaskRunner pid=2074004) 'mode': 'async', (TaskRunner pid=2074004) 'multi_stage_wake_up': False, (TaskRunner pid=2074004) 'multi_turn': {'_target_': 'verl.workers.config.MultiTurnConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'format': 'hermes', (TaskRunner pid=2074004) 'interaction_config_path': None, (TaskRunner pid=2074004) 'max_assistant_turns': None, (TaskRunner pid=2074004) 'max_parallel_calls': 1, (TaskRunner pid=2074004) 'max_tool_response_length': 256, (TaskRunner pid=2074004) 'max_user_turns': None, (TaskRunner pid=2074004) 'num_repeat_rollouts': None, (TaskRunner pid=2074004) 'tokenization_sanity_check_mode': 'strict', (TaskRunner pid=2074004) 'tool_config_path': None, (TaskRunner pid=2074004) 'tool_response_truncate_side': 'middle', (TaskRunner pid=2074004) 'use_inference_chat_template': False}, (TaskRunner pid=2074004) 'n': 8, (TaskRunner pid=2074004) 'name': 'vllm', (TaskRunner pid=2074004) 'over_sample_rate': 0, (TaskRunner pid=2074004) 'pipeline_model_parallel_size': 1, (TaskRunner pid=2074004) 'profiler': (TaskRunner pid=2074004) {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'prometheus': {'_target_': 'verl.workers.config.PrometheusConfig', (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'file': '/tmp/ray/session_latest/metrics/prometheus/prometheus.yml', (TaskRunner pid=2074004) 'port': 9090, (TaskRunner pid=2074004) 'served_model_name': 'Qwen/Qwen3-4B'}, (TaskRunner pid=2074004) 'prompt_length': 2048, (TaskRunner pid=2074004) 'quantization': None, (TaskRunner pid=2074004) 'quantization_config_file': None, (TaskRunner pid=2074004) 'response_length': 8192, (TaskRunner pid=2074004) 'scheduling_policy': 'fcfs', (TaskRunner pid=2074004) 'skip_dump_dir': '/tmp/rollout_dump', (TaskRunner pid=2074004) 'skip_rollout': False, (TaskRunner pid=2074004) 'skip_tokenizer_init': True, (TaskRunner pid=2074004) 'temperature': 1.0, (TaskRunner pid=2074004) 'tensor_model_parallel_size': 2, (TaskRunner pid=2074004) 'top_k': -1, (TaskRunner pid=2074004) 'top_p': 1.0, (TaskRunner pid=2074004) 'trace': {'_target_': 'verl.workers.config.TraceConfig', (TaskRunner pid=2074004) 'backend': None, (TaskRunner pid=2074004) 'max_samples_per_step_per_worker': None, (TaskRunner pid=2074004) 'token2text': False}, (TaskRunner pid=2074004) 'update_weights_bucket_megabytes': 512, (TaskRunner pid=2074004) 'val_kwargs': {'_target_': 'verl.workers.config.SamplingConfig', (TaskRunner pid=2074004) 'do_sample': True, (TaskRunner pid=2074004) 'n': 16, (TaskRunner pid=2074004) 'temperature': 0.6, (TaskRunner pid=2074004) 'top_k': -1, (TaskRunner pid=2074004) 'top_p': 0.95}}}, (TaskRunner pid=2074004) 'algorithm': {'_target_': 'verl.trainer.config.AlgoConfig', (TaskRunner pid=2074004) 'adv_estimator': 'grpo', (TaskRunner pid=2074004) 'gamma': 1.0, (TaskRunner pid=2074004) 'kl_ctrl': {'_target_': 'verl.trainer.config.KLControlConfig', (TaskRunner pid=2074004) 'horizon': 10000, (TaskRunner pid=2074004) 'kl_coef': 0.001, (TaskRunner pid=2074004) 'target_kl': 0.1, (TaskRunner pid=2074004) 'type': 'fixed'}, (TaskRunner pid=2074004) 'kl_penalty': 'kl', (TaskRunner pid=2074004) 'lam': 1.0, (TaskRunner pid=2074004) 'norm_adv_by_std_in_grpo': False, (TaskRunner pid=2074004) 'pf_ppo': {'reweight_method': 'pow', 'weight_pow': 2.0}, (TaskRunner pid=2074004) 'rollout_correction': {'bypass_mode': False, (TaskRunner pid=2074004) 'loss_type': 'ppo_clip', (TaskRunner pid=2074004) 'rollout_is': 'token', (TaskRunner pid=2074004) 'rollout_is_batch_normalize': False, (TaskRunner pid=2074004) 'rollout_is_threshold': 2.0, (TaskRunner pid=2074004) 'rollout_rs': None, (TaskRunner pid=2074004) 'rollout_rs_threshold': None}, (TaskRunner pid=2074004) 'use_kl_in_reward': False, (TaskRunner pid=2074004) 'use_pf_ppo': False}, (TaskRunner pid=2074004) 'critic': {'_target_': 'verl.workers.config.FSDPCriticConfig', (TaskRunner pid=2074004) 'checkpoint': {'_target_': 'verl.trainer.config.CheckpointConfig', (TaskRunner pid=2074004) 'async_save': False, (TaskRunner pid=2074004) 'load_contents': ['model', 'optimizer', 'extra'], (TaskRunner pid=2074004) 'save_contents': ['model', 'optimizer', 'extra']}, (TaskRunner pid=2074004) 'cliprange_value': 0.5, (TaskRunner pid=2074004) 'data_loader_seed': 42, (TaskRunner pid=2074004) 'enable': None, (TaskRunner pid=2074004) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'forward_micro_batch_size': None, (TaskRunner pid=2074004) 'forward_micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'grad_clip': 1.0, (TaskRunner pid=2074004) 'loss_agg_mode': 'token-mean', (TaskRunner pid=2074004) 'model': {'_target_': 'verl.workers.config.FSDPCriticModelCfg', (TaskRunner pid=2074004) 'enable_activation_offload': False, (TaskRunner pid=2074004) 'enable_gradient_checkpointing': True, (TaskRunner pid=2074004) 'external_lib': None, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'entropy_checkpointing': False, (TaskRunner pid=2074004) 'entropy_from_logits_with_chunking': False, (TaskRunner pid=2074004) 'forward_only': False, (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'full_determinism': False, (TaskRunner pid=2074004) 'model_dtype': 'fp32', (TaskRunner pid=2074004) 'offload_policy': False, (TaskRunner pid=2074004) 'optimizer_offload': False, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'seed': 42, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_orig_params': False, (TaskRunner pid=2074004) 'use_torch_compile': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'lora_alpha': 16, (TaskRunner pid=2074004) 'lora_rank': 0, (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': 'Qwen/Qwen3-8B', (TaskRunner pid=2074004) 'target_modules': 'all-linear', (TaskRunner pid=2074004) 'tiled_mlp': {'enabled': False, 'num_shards': 4}, (TaskRunner pid=2074004) 'tokenizer_path': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_remove_padding': False, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'optim': {'_target_': 'verl.workers.config.FSDPOptimizerConfig', (TaskRunner pid=2074004) 'betas': [0.9, 0.999], (TaskRunner pid=2074004) 'clip_grad': 1.0, (TaskRunner pid=2074004) 'lr': 1e-05, (TaskRunner pid=2074004) 'lr_scheduler_type': 'constant', (TaskRunner pid=2074004) 'lr_warmup_steps': -1, (TaskRunner pid=2074004) 'lr_warmup_steps_ratio': 0.0, (TaskRunner pid=2074004) 'min_lr_ratio': 0.0, (TaskRunner pid=2074004) 'num_cycles': 0.5, (TaskRunner pid=2074004) 'optimizer': 'AdamW', (TaskRunner pid=2074004) 'optimizer_impl': 'torch.optim', (TaskRunner pid=2074004) 'override_optimizer_config': None, (TaskRunner pid=2074004) 'total_training_steps': -1, (TaskRunner pid=2074004) 'warmup_style': None, (TaskRunner pid=2074004) 'weight_decay': 0.01}, (TaskRunner pid=2074004) 'ppo_epochs': 1, (TaskRunner pid=2074004) 'ppo_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'ppo_micro_batch_size': None, (TaskRunner pid=2074004) 'ppo_micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'ppo_mini_batch_size': 8, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'rollout_n': 8, (TaskRunner pid=2074004) 'shuffle': False, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False}, (TaskRunner pid=2074004) 'custom_reward_function': {'name': 'compute_score', (TaskRunner pid=2074004) 'path': '/mnt/mole/SDPO/L2T/verl/utils/reward_score/feedback/__init__.py'}, (TaskRunner pid=2074004) 'data': {'apply_chat_template_kwargs': {'enable_thinking': False}, (TaskRunner pid=2074004) 'custom_cls': {'name': None, 'path': None}, (TaskRunner pid=2074004) 'datagen': {'name': None, 'path': None}, (TaskRunner pid=2074004) 'dataloader_num_workers': 8, (TaskRunner pid=2074004) 'filter_overlong_prompts': True, (TaskRunner pid=2074004) 'filter_overlong_prompts_workers': 1, (TaskRunner pid=2074004) 'image_key': 'images', (TaskRunner pid=2074004) 'image_patch_size': 14, (TaskRunner pid=2074004) 'max_prompt_length': 2048, (TaskRunner pid=2074004) 'max_response_length': 8192, (TaskRunner pid=2074004) 'prompt_key': 'prompt', (TaskRunner pid=2074004) 'return_full_prompt': False, (TaskRunner pid=2074004) 'return_multi_modal_inputs': True, (TaskRunner pid=2074004) 'return_raw_chat': True, (TaskRunner pid=2074004) 'return_raw_input_ids': False, (TaskRunner pid=2074004) 'reward_fn_key': 'data_source', (TaskRunner pid=2074004) 'sampler': {'class_name': None, 'class_path': None}, (TaskRunner pid=2074004) 'seed': None, (TaskRunner pid=2074004) 'shuffle': True, (TaskRunner pid=2074004) 'tokenizer': None, (TaskRunner pid=2074004) 'tool_config_path': None, (TaskRunner pid=2074004) 'train_batch_size': 32, (TaskRunner pid=2074004) 'train_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/train.parquet'], (TaskRunner pid=2074004) 'train_max_samples': 3200, (TaskRunner pid=2074004) 'truncation': 'error', (TaskRunner pid=2074004) 'trust_remote_code': True, (TaskRunner pid=2074004) 'use_shm': False, (TaskRunner pid=2074004) 'val_batch_size': None, (TaskRunner pid=2074004) 'val_files': ['/mnt/mole/SDPO/L2T/datasets/sciknoweval/physics/test.parquet'], (TaskRunner pid=2074004) 'val_max_samples': -1, (TaskRunner pid=2074004) 'validation_shuffle': False, (TaskRunner pid=2074004) 'video_key': 'videos'}, (TaskRunner pid=2074004) 'global_profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'global_tool_config': {'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'controller_nsight_options': {'cuda-graph-trace': 'graph', (TaskRunner pid=2074004) 'cuda-memory-usage': 'true', (TaskRunner pid=2074004) 'trace': 'cuda,nvtx,cublas,ucx'}, (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'worker_nsight_options': {'capture-range': 'cudaProfilerApi', (TaskRunner pid=2074004) 'capture-range-end': None, (TaskRunner pid=2074004) 'cuda-graph-trace': 'graph', (TaskRunner pid=2074004) 'cuda-memory-usage': 'true', (TaskRunner pid=2074004) 'kill': 'none', (TaskRunner pid=2074004) 'trace': 'cuda,nvtx,cublas,ucx'}}, (TaskRunner pid=2074004) 'torch_memory': {'context': 'all', (TaskRunner pid=2074004) 'kw_args': {}, (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'stacks': 'all', (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}, (TaskRunner pid=2074004) 'profile_continuous_steps': False, (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'steps': None, (TaskRunner pid=2074004) 'tool': None}, (TaskRunner pid=2074004) 'max_model_len': 10240, (TaskRunner pid=2074004) 'ray_kwargs': {'ray_init': {'_temp_dir': '/tmp/ray_q3g_physics_rlsd_tr_Qwen3_4B', (TaskRunner pid=2074004) 'include_dashboard': False, (TaskRunner pid=2074004) 'num_cpus': None, (TaskRunner pid=2074004) 'runtime_env': {'env_vars': {'EXPERIMENT': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'TASK': 'datasets/sciknoweval/physics', (TaskRunner pid=2074004) 'USER': 'root'}}}, (TaskRunner pid=2074004) 'timeline_json_file': None}, (TaskRunner pid=2074004) 'reward_manager': {'_target_': 'verl.trainer.config.config.RewardManagerConfig', (TaskRunner pid=2074004) 'module': {'_target_': 'verl.trainer.config.config.ModuleConfig', (TaskRunner pid=2074004) 'name': 'custom_reward_manager', (TaskRunner pid=2074004) 'path': None}, (TaskRunner pid=2074004) 'name': 'naive', (TaskRunner pid=2074004) 'source': 'register'}, (TaskRunner pid=2074004) 'reward_model': {'enable': False, (TaskRunner pid=2074004) 'enable_resource_pool': False, (TaskRunner pid=2074004) 'forward_max_token_len_per_gpu': 32768, (TaskRunner pid=2074004) 'launch_reward_fn_async': False, (TaskRunner pid=2074004) 'max_length': None, (TaskRunner pid=2074004) 'micro_batch_size': None, (TaskRunner pid=2074004) 'micro_batch_size_per_gpu': None, (TaskRunner pid=2074004) 'model': {'external_lib': None, (TaskRunner pid=2074004) 'fsdp_config': {'_target_': 'verl.workers.config.FSDPEngineConfig', (TaskRunner pid=2074004) 'forward_prefetch': False, (TaskRunner pid=2074004) 'fsdp_size': -1, (TaskRunner pid=2074004) 'param_offload': False, (TaskRunner pid=2074004) 'reshard_after_forward': True, (TaskRunner pid=2074004) 'wrap_policy': {'min_num_params': 0}}, (TaskRunner pid=2074004) 'input_tokenizer': 'Qwen/Qwen3-4B', (TaskRunner pid=2074004) 'override_config': {}, (TaskRunner pid=2074004) 'path': '~/models/FsfairX-LLaMA3-RM-v0.1', (TaskRunner pid=2074004) 'trust_remote_code': False, (TaskRunner pid=2074004) 'use_fused_kernels': False, (TaskRunner pid=2074004) 'use_remove_padding': False, (TaskRunner pid=2074004) 'use_shm': False}, (TaskRunner pid=2074004) 'n_gpus_per_node': 8, (TaskRunner pid=2074004) 'nnodes': 0, (TaskRunner pid=2074004) 'num_workers': 1, (TaskRunner pid=2074004) 'profiler': {'_target_': 'verl.utils.profiler.ProfilerConfig', (TaskRunner pid=2074004) 'all_ranks': False, (TaskRunner pid=2074004) 'enable': False, (TaskRunner pid=2074004) 'ranks': [], (TaskRunner pid=2074004) 'save_path': 'outputs/profile', (TaskRunner pid=2074004) 'tool': None, (TaskRunner pid=2074004) 'tool_config': {'npu': {'_target_': 'verl.utils.profiler.config.NPUToolConfig', (TaskRunner pid=2074004) 'analysis': True, (TaskRunner pid=2074004) 'contents': [], (TaskRunner pid=2074004) 'discrete': False, (TaskRunner pid=2074004) 'level': 'level0'}, (TaskRunner pid=2074004) 'nsys': {'_target_': 'verl.utils.profiler.config.NsightToolConfig', (TaskRunner pid=2074004) 'discrete': False}, (TaskRunner pid=2074004) 'torch': {'_target_': 'verl.utils.profiler.config.TorchProfilerToolConfig', (TaskRunner pid=2074004) 'step_end': None, (TaskRunner pid=2074004) 'step_start': 0}, (TaskRunner pid=2074004) 'torch_memory': {'_target_': 'verl.utils.profiler.config.TorchMemoryToolConfig', (TaskRunner pid=2074004) 'stack_depth': 32, (TaskRunner pid=2074004) 'trace_alloc_max_entries': 100000}}}, (TaskRunner pid=2074004) 'reward_loop_class_name': None, (TaskRunner pid=2074004) 'reward_loop_module_path': None, (TaskRunner pid=2074004) 'reward_loop_source': 'register', (TaskRunner pid=2074004) 'reward_manager': 'naive', (TaskRunner pid=2074004) 'rollout': {'_target_': 'verl.workers.config.RolloutConfig', (TaskRunner pid=2074004) 'cudagraph_capture_sizes': None, (TaskRunner pid=2074004) 'data_parallel_size': 1, (TaskRunner pid=2074004) 'disable_log_stats': True, (TaskRunner pid=2074004) 'dtype': 'bfloat16', (TaskRunner pid=2074004) 'enable_chunked_prefill': True, (TaskRunner pid=2074004) 'enable_prefix_caching': True, (TaskRunner pid=2074004) 'enforce_eager': True, (TaskRunner pid=2074004) 'engine_kwargs': {}, (TaskRunner pid=2074004) 'expert_parallel_size': 1, (TaskRunner pid=2074004) 'free_cache_engine': True, (TaskRunner pid=2074004) 'gpu_memory_utilization': 0.5, (TaskRunner pid=2074004) 'limit_images': None, (TaskRunner pid=2074004) 'load_format': 'auto', (TaskRunner pid=2074004) 'max_model_len': None, (TaskRunner pid=2074004) 'max_num_batched_tokens': 8192, (TaskRunner pid=2074004) 'max_num_seqs': 1024, (TaskRunner pid=2074004) 'name': '???', (TaskRunner pid=2074004) 'prompt_length': 2048, (TaskRunner pid=2074004) 'response_length': 2048, (TaskRunner pid=2074004) 'skip_tokenizer_init': False, (TaskRunner pid=2074004) 'tensor_model_parallel_size': 2}, (TaskRunner pid=2074004) 'sandbox_fusion': {'max_concurrent': 64, (TaskRunner pid=2074004) 'memory_limit_mb': 1024, (TaskRunner pid=2074004) 'url': None}, (TaskRunner pid=2074004) 'strategy': 'fsdp', (TaskRunner pid=2074004) 'ulysses_sequence_parallel_size': 1, (TaskRunner pid=2074004) 'use_dynamic_bsz': False, (TaskRunner pid=2074004) 'use_reward_loop': False}, (TaskRunner pid=2074004) 'trainer': {'balance_batch': True, (TaskRunner pid=2074004) 'critic_warmup': 0, (TaskRunner pid=2074004) 'default_hdfs_dir': None, (TaskRunner pid=2074004) 'default_local_dir': '/mnt/mole/SDPO/L2T/checkpoints/datasets/sciknoweval/physics/qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'del_local_ckpt_after_load': False, (TaskRunner pid=2074004) 'device': 'cuda', (TaskRunner pid=2074004) 'esi_redundant_time': 0, (TaskRunner pid=2074004) 'experiment_name': 'qwen3gen-physics-RLSD_TR-Qwen-Qwen3-4B-mbs8-decay0-tr0.1-train32-rollout8-lr1e-6-vllm0.8', (TaskRunner pid=2074004) 'group_name': 'QWEN3-RLSD-TR-GRPO-matched-generalization', (TaskRunner pid=2074004) 'log_val_generations': 0, (TaskRunner pid=2074004) 'logger': ['console', 'wandb'], (TaskRunner pid=2074004) 'max_actor_ckpt_to_keep': 1, (TaskRunner pid=2074004) 'max_critic_ckpt_to_keep': None, (TaskRunner pid=2074004) 'n_gpus_per_node': 8, (TaskRunner pid=2074004) 'nnodes': 1, (TaskRunner pid=2074004) 'project_name': 'SDPO-root', (TaskRunner pid=2074004) 'ray_wait_register_center_timeout': 300, (TaskRunner pid=2074004) 'resume_from_path': None, (TaskRunner pid=2074004) 'resume_mode': 'auto', (TaskRunner pid=2074004) 'rollout_data_dir': None, (TaskRunner pid=2074004) 'save_freq': 10, (TaskRunner pid=2074004) 'test_freq': 10, (TaskRunner pid=2074004) 'total_epochs': 30, (TaskRunner pid=2074004) 'total_training_steps': 100, (TaskRunner pid=2074004) 'use_legacy_worker_impl': 'auto', (TaskRunner pid=2074004) 'val_before_train': False, (TaskRunner pid=2074004) 'val_only': False, (TaskRunner pid=2074004) 'validation_data_dir': None}, (TaskRunner pid=2074004) 'transfer_queue': {'enable': False}, (TaskRunner pid=2074004) 'vars': {'ckpt_dir': '/capstor/scratch/cscs/root/ttrl_runs/datasets/sciknoweval/physics', (TaskRunner pid=2074004) 'dir': '/users/root/SDPO', (TaskRunner pid=2074004) 'log_dir': '/users/root/output', (TaskRunner pid=2074004) 'task': 'datasets/sciknoweval/physics'}} (TaskRunner pid=2074004) [validate_config] All configuration checks passed successfully! (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/trainer/main_ppo.py:326: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2074004) use_critic=need_critic(config), (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2074004) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (TaskRunner pid=2074004) Using dataset class: RLHFDataset (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (TaskRunner pid=2074004) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (TaskRunner pid=2074004) dataset len: 720 (TaskRunner pid=2074004) Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2074004) WARNING:2026-07-02 08:56:38,173:Setting TOKENIZERS_PARALLELISM=false for forked processes. (TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 0%| | 0/720 [00:00 (TaskRunner pid=2074004) bind role actor_rollout_ref method chat_completion to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method generate to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method get_zeromq_address to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method sleep to class .WorkerDict'> (TaskRunner pid=2074004) bind role actor_rollout_ref method wake_up to class .WorkerDict'> (TaskRunner pid=2074004) Filtering prompts longer than 2048 tokens (num_proc=1): 100%|██████████| 80/80 [00:00<00:00, 128.23 examples/s] (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/trainer/ppo/ray_trainer.py:354: UserWarning: Disabled critic as algorithm.adv_estimator != gae. If it is not intended, please set critic.enable=True (TaskRunner pid=2074004) self.use_critic = need_critic(self.config) (WorkerDict pid=2074388) [W702 08:56:47.781903761 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56733 (errno: 97 - Address family not supported by protocol). (WorkerDict pid=2074385) [Gloo] Rank 0 is connected to 7 peer ranks. Expected number of connected peer ranks is : 7 (WorkerDict pid=2074386) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2074386) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2074387) [W702 08:56:47.990315263 socket.cpp:767] [c10d] The client socket cannot be initialized to connect to [::ffff:198.19.44.212]:56733 (errno: 97 - Address family not supported by protocol). [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.) (WorkerDict pid=2074386) `torch_dtype` is deprecated! Use `dtype` instead! (WorkerDict pid=2074386) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3ForCausalLM is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2074386) Flash Attention 2 only supports torch.float16 and torch.bfloat16 dtypes, but the current dype in Qwen3Model is torch.float32. You should run training or inference using Automatic Mixed-Precision via the `with torch.autocast(device_type='torch_device'):` decorator, or load the model with the `dtype` argument. Example: `model = AutoModel.from_pretrained("openai/whisper-tiny", attn_implementation="flash_attention_2", dtype=torch.float16)` (WorkerDict pid=2074391) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2074385) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2074385) Total steps: 100, num_warmup_steps: 10 (WorkerDict pid=2074385) Actor use_remove_padding=True (WorkerDict pid=2074385) Actor use_fused_kernels=False (WorkerDict pid=2074385) Actor use_prefix_grouper=False (WorkerDict pid=2074385) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2074385) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (WorkerDict pid=2074385) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (WorkerDict pid=2074385) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2074387) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.99s/it] [repeated 12x across cluster] (WorkerDict pid=2074385) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:05<00:02, 2.90s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:05<00:00, 1.95s/it] [repeated 2x across cluster] (WorkerDict pid=2074387) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.01s/it] [repeated 4x across cluster] (WorkerDict pid=2074385) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (WorkerDict pid=2074385) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (WorkerDict pid=2074385) [Gloo] Rank 0 is connected to 3 peer ranks. Expected number of connected peer ranks is : 3 (WorkerDict pid=2074386) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. [repeated 7x across cluster] (WorkerDict pid=2074386) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) [repeated 7x across cluster] (WorkerDict pid=2074391) /mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py:675: FutureWarning: FSDP.state_dict_type() and FSDP.set_state_dict_type() are being deprecated. Please use APIs, get_state_dict() and set_state_dict(), which can support different parallelisms, FSDP1, FSDP2, DDP. API doc: https://pytorch.org/docs/stable/distributed.checkpoint.html#torch.distributed.checkpoint.state_dict.get_state_dict .Tutorial: https://pytorch.org/tutorials/recipes/distributed_checkpoint_recipe.html . (WorkerDict pid=2074391) warnings.warn( (WorkerDict pid=2074385) reference model: Qwen/Qwen3-4B (WorkerDict pid=2074391) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 8x across cluster] (WorkerDict pid=2074391) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 8x across cluster] (WorkerDict pid=2074385) Model config after override: Qwen3Config { (WorkerDict pid=2074385) "architectures": [ (WorkerDict pid=2074385) "Qwen3ForCausalLM" (WorkerDict pid=2074385) ], (WorkerDict pid=2074385) "attention_bias": false, (WorkerDict pid=2074385) "attention_dropout": 0.0, (WorkerDict pid=2074385) "dtype": "bfloat16", (WorkerDict pid=2074385) "eos_token_id": 151645, (WorkerDict pid=2074385) "head_dim": 128, (WorkerDict pid=2074385) "hidden_act": "silu", (WorkerDict pid=2074385) "hidden_size": 2560, (WorkerDict pid=2074385) "initializer_range": 0.02, (WorkerDict pid=2074385) "intermediate_size": 9728, (WorkerDict pid=2074385) "layer_types": [ (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention", (WorkerDict pid=2074385) "full_attention" (WorkerDict pid=2074385) ], (WorkerDict pid=2074385) "max_position_embeddings": 40960, (WorkerDict pid=2074385) "max_window_layers": 36, (WorkerDict pid=2074385) "model_type": "qwen3", (WorkerDict pid=2074385) "num_attention_heads": 32, (WorkerDict pid=2074385) "num_hidden_layers": 36, (WorkerDict pid=2074385) "num_key_value_heads": 8, (WorkerDict pid=2074385) "pad_token_id": 151643, (WorkerDict pid=2074385) "rms_norm_eps": 1e-06, (WorkerDict pid=2074385) "rope_scaling": null, (WorkerDict pid=2074385) "rope_theta": 1000000, (WorkerDict pid=2074385) "sliding_window": null, (WorkerDict pid=2074385) "tie_word_embeddings": true, (WorkerDict pid=2074385) "transformers_version": "4.57.1", (WorkerDict pid=2074385) "use_cache": true, (WorkerDict pid=2074385) "use_sliding_window": false, (WorkerDict pid=2074385) "vocab_size": 151936 (WorkerDict pid=2074385) } (WorkerDict pid=2074385) (WorkerDict pid=2074387) [Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0 [repeated 23x across cluster] (WorkerDict pid=2074391) Loading checkpoint shards: 0%| | 0/3 [00:00, policies=[functools.partial(, transformer_layer_cls={})]) (WorkerDict pid=2074385) Ref use_remove_padding=True (WorkerDict pid=2074385) Ref use_fused_kernels=False (WorkerDict pid=2074385) Ref use_prefix_grouper=False (WorkerDict pid=2074392) Monkey patch _flash_attention_forward in transformers.integrations.flash_attention [repeated 7x across cluster] (WorkerDict pid=2074392) Skipping monkey patch for Qwen3ForCausalLM as use_fused_kernels is False or fused_kernels_backend is torch [repeated 7x across cluster] (TaskRunner pid=2074004) /mnt/mole/SDPO/L2T/verl/utils/profiler/config.py:52: UserWarning: Torch profiler tool config is not fully supported now. (TaskRunner pid=2074004) warnings.warn("Torch profiler tool config is not fully supported now.", stacklevel=1) (WorkerDict pid=2074392) Loading checkpoint shards: 67%|██████▋ | 2/3 [00:06<00:03, 3.02s/it] [repeated 14x across cluster] (WorkerDict pid=2074392) Loading checkpoint shards: 100%|██████████| 3/3 [00:06<00:00, 2.04s/it] [repeated 6x across cluster] (vLLMHttpServer pid=2075282) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing (vLLMHttpServer pid=2075282) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) (vLLMHttpServer pid=2075282) ['serve', (vLLMHttpServer pid=2075282) 'Qwen/Qwen3-4B', (vLLMHttpServer pid=2075282) '--dtype', (vLLMHttpServer pid=2075282) 'bfloat16', (vLLMHttpServer pid=2075282) '--load_format', (vLLMHttpServer pid=2075282) 'dummy', (vLLMHttpServer pid=2075282) '--trust_remote_code', (vLLMHttpServer pid=2075282) '--max_model_len', (vLLMHttpServer pid=2075282) '40960', (vLLMHttpServer pid=2075282) '--max_num_seqs', (vLLMHttpServer pid=2075282) '1024', (vLLMHttpServer pid=2075282) '--enable_chunked_prefill', (vLLMHttpServer pid=2075282) '--max_num_batched_tokens', (vLLMHttpServer pid=2075282) '10240', (vLLMHttpServer pid=2075282) '--enable_prefix_caching', (vLLMHttpServer pid=2075282) '--enable_sleep_mode', (vLLMHttpServer pid=2075282) '--logprobs_mode', (vLLMHttpServer pid=2075282) 'processed_logprobs', (vLLMHttpServer pid=2075282) '--disable_custom_all_reduce', (vLLMHttpServer pid=2075282) '--gpu_memory_utilization', (vLLMHttpServer pid=2075282) '0.8', (vLLMHttpServer pid=2075282) '--disable_log_stats', (vLLMHttpServer pid=2075282) '--tensor_parallel_size', (vLLMHttpServer pid=2075282) '2', (vLLMHttpServer pid=2075282) '--seed', (vLLMHttpServer pid=2075282) '0', (vLLMHttpServer pid=2075282) '--override_generation_config', (vLLMHttpServer pid=2075282) '{"temperature": 1.0, "top_k": -1, "top_p": 1.0, "repetition_penalty": 1.0, ' (vLLMHttpServer pid=2075282) '"max_new_tokens": 8192}', (vLLMHttpServer pid=2075282) '--hf_overrides', (vLLMHttpServer pid=2075282) '{}', (vLLMHttpServer pid=2075282) '--scheduling_policy', (vLLMHttpServer pid=2075282) 'fcfs'] (vLLMHttpServer pid=2075282) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. (vLLMHttpServer pid=2075282) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. (vLLMHttpServer pid=2075282) WARNING 07-02 08:57:45 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned (WorkerDict pid=2074385) WARNING 07-02 08:57:52 [worker_base.py:301] Missing `shared_worker_lock` argument from executor. This argument is needed for mm_processor_cache_type='shm'. (vLLMHttpServer pid=2075287) WARNING 07-02 08:57:46 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: In a Ray actor and can only be spawned [repeated 3x across cluster] (WorkerDict pid=2074389) NCCL version 2.27.5+cuda12.9 (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] fx graph cache unable to load compiled graph (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] Traceback (most recent call last): (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] File "/mnt/mole/SDPO/L2T/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py", line 1017, in iterate_over_candidates (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] with open(os.path.join(subdir, path), "rb") as f: (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (WorkerDict pid=2074392) [rank7]:W0702 08:58:04.240000 2074392 /mnt/mole/SDPO/self-distillation-analysis/.venv/lib/python3.12/site-packages/torch/_inductor/codecache.py:1021] [0/0] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/torchinductor_root/aotautograd/aqwmczjqsm4pvwfimkpe3rswyj2xfwbwedbijxvwd7tfo5n2eqnk/.2074388.133380852872896.tmp' (vLLMHttpServer pid=2075287) /mnt/mole/SDPO/L2T/verl/utils/tokenizer.py:107: UserWarning: Failed to create processor: Unsupported processor type: Qwen2TokenizerFast. This may affect multimodal processing [repeated 3x across cluster] (vLLMHttpServer pid=2075287) warnings.warn(f"Failed to create processor: {e}. This may affect multimodal processing", stacklevel=1) [repeated 3x across cluster] (vLLMHttpServer pid=2075287) Using blocking ray.get inside async actor. This blocks the event loop. Please use `await` on object ref with asyncio.gather if you want to yield execution to the event loop instead. [repeated 3x across cluster] (vLLMHttpServer pid=2075287) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored. [repeated 3x across cluster] (WorkerDict pid=2074385) 2026-07-02 08:58:08,146 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (WorkerDict pid=2074386) 2026-07-02 08:58:08,165 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends (WorkerDict pid=2074385) Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 0%| | 0/51 [00:00